THE DATABASE FOR MULTIFACTORIAL UAV ACCURACY ASSESSMENTS

Due to the increasing range of work carried out with UAV in recent years, the importance of final product accuracy appreciates. However, obtaining survey-grade accuracy requires to perform bundle adjustment processes that could be affected by multiple factors like unstable camera calibration, a correlation between interior and exterior orientation, insufficient georeferenced information, and software settings. During the project, multi-variant flight over the test field was conducted. The flights were performed with a fixedwing airframe equipped with PPK receiver on-board. Based on the conducted flights, the database for multifactorial data sets has been prepared. The database containing hundreds of independent adjustment variants which differ as follows: georeferencing method, flight configuration, additional camera calibration corrections, tie points filtering, and a priori accuracy settings. The database allowed to investigate the separate influence of each factor on the final results using ANOVA statistical models. 1. INTORODUCTION The use of drones in surveying is developing rapidly and areas of UAVs application are getting the more and more various [Nex, 2014]. Some types of surveys, such as natural hazards inventorying, vegetation monitoring, cultural heritages mapping and many others, need more good interpretation condition, then very high geometrical trueness and precision. At the same time, there are surveys such as cadastral mapping or base map elaboration which require accuracy on a few centimetre level [Kurczynski, Bakuła, 2016, Kędzierski, Fryskowska, 2016, van Hinsberg, Cramer, 2013]. Among many publications on UAV-based aerial surveying, relatively little of them are concerned with matters of factors negatively affecting the accuracy. The problem is not trivial due to the fact, that the fundamental condition for achieving a correct rays intersection is the knowledge of external and internal cameras orientation parameters. Unfortunately, the estimation of those parameters is affected due to their high correlation. On this account the classical photogrammetry capture images using metric cameras which internal is orientation is performed autonomously in laboratory or in field test. This way does not work in UAV survey application due to using light consumer-grade cameras with unstable internal orientation. In practice, for non-metric cameras used in UAV a method of self-calibration is considered as the most reliable due to the issue of poor stability of the internal orientation. In this process additional unknowns like principal distance, principal point and distortion are determined simultaneously with the exterior orientation that makes it very sensitive to the structure of image block and input observations and raises the question about actual accuracy and reliability [Luhman, 2015]. What is more software developers have recognized the impact of Brown model deficiency for non-metric cameras distortion. To compensate it they added additional corrections models such as rolling shutter compensation or recently added to Agisoft Metashape 1.6 Fourier additional correction model with 96 parameters. All of these factors raised concerns on over parametrization and increased dependencies between adjustment parameters and final results. The issue of poor stability of internal orientation of cameras was described by Cramer and co-authors [Cramer 2017]. The research has showed out that in some cameras the focal length and principal point position are changing in several dozen of microns, although between calibration the camera was not operationally used. It is worth underlining that also thermal conditions impact the parameters of interior orientation [Daakir, 2019]. The standard method of image orientation, called indirect georeferencing, is a rigorous solution by bundle adjustment using Ground Control Points. The need to measure GCPs significantly increases the work time and the cost of UAVbased mapping. The fundamental question is about the minimal number of GCPs and its localization in elaborated area which are need to achieve a good precision of image orientation. Most papers express the view that the GCPs number should be between 3 and dozen, and suggest 6 GCPs as an optimal number [James, 2017]. Several authors advise more than 2 GCPs per 100 photos [Sanz-Ablanedo, 2018]. Due to various characteristic of mapped area (size, shape, relief, land cover) there are no universal rules about optimal number of GCPs and their spatial distribution. The direct georeferencing of UAV images has been developed for some years. Nowadays, many vendors install survey-grade RTK-GNSS receiver on board the UAV. The application of cameras pose with centimetre accuracy in SFM workflow opens the door to UAV elaboration without GCPs. There are some experiences that confirm the expectation [Gerke, Stöcker, 2019]. During the project, multi-variant flight over the test field was conducted. The flights were performed with a fixed-wing airframe equipped with PPK receiver on-board. Based on the conducted flight, many data sets have been prepared, which differ as follows: • flight configuration, • georeferencing method: (1) GCP + approximate EO, (2) GCP + precise EO, (3) precise EO without GCP, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B5-2020, 2020 XXIV ISPRS Congress (2020 edition) This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B5-2020-163-2020 | © Authors 2020. CC BY 4.0 License. 163 • additional correction model for camera calibration, • key point/tie points limit: (1) without tie points, (2) 10k/500, (3) 40k/4k, • tie point filtering, • a priori accuracy settings. The main aim of the research was to create database based on the conducted flight. The database containing hundreds of independent adjustment variants which allowed to investigate the separate influence of each factor on to final results. Based on the obtain database multiple statistical analysis were conducted to investigate: • Influence of using Fourier additional correction model, • Influence of georeferencing method on correlation between interior and exterior orientation • Influence of tie points count and distribution • Influence of observation weighting b) Ground Control Points c) Test field terrain altitude Figure 1. Test field overview 2. TEST FIELD Test field used for the investigation was located in residential area in Bochnia city covered area of average 350 m x 400 m. The size of the test field enable multi-variant coverage within one flight with fixed wing UAV. The area of the test field is characterized by a large number of internal roads, several storey blocks and terrain altitude differences up to 35 meters. There was 70 check points distributed across entire area (Figure 1). 11 points were marked and located on building roofs and 59 were natural marked ground points. As a natural marked, points like parking lines and wells were adopted while as a roof points special targets made of PVC were used (Figure 2). Check points were measured in three session from June 2018 to December 2018 using RTK and RTN method. Based on these measurements; the accuracy of the points on 2 cm horizontal and 3 cm vertical level was determined (Table 1).


INTORODUCTION
The use of drones in surveying is developing rapidly and areas of UAVs application are getting the more and more various [Nex, 2014]. Some types of surveys, such as natural hazards inventorying, vegetation monitoring, cultural heritages mapping and many others, need more good interpretation condition, then very high geometrical trueness and precision. At the same time, there are surveys such as cadastral mapping or base map elaboration which require accuracy on a few centimetre level [Kurczynski, Bakuła, 2016, Kędzierski, Fryskowska, 2016, van Hinsberg, Cramer, 2013.
Among many publications on UAV-based aerial surveying, relatively little of them are concerned with matters of factors negatively affecting the accuracy. The problem is not trivial due to the fact, that the fundamental condition for achieving a correct rays intersection is the knowledge of external and internal cameras orientation parameters. Unfortunately, the estimation of those parameters is affected due to their high correlation. On this account the classical photogrammetry capture images using metric cameras which internal is orientation is performed autonomously in laboratory or in field test. This way does not work in UAV survey application due to using light consumer-grade cameras with unstable internal orientation. In practice, for non-metric cameras used in UAV a method of self-calibration is considered as the most reliable due to the issue of poor stability of the internal orientation. In this process additional unknowns like principal distance, principal point and distortion are determined simultaneously with the exterior orientation that makes it very sensitive to the structure of image block and input observations and raises the question about actual accuracy and reliability [Luhman, 2015]. What is more software developers have recognized the impact of Brown model deficiency for non-metric cameras distortion. To compensate it they added additional corrections models such as rolling shutter compensation or recently added to Agisoft Metashape 1.6 Fourier additional correction model with 96 parameters. All of these factors raised concerns on over parametrization and increased dependencies between adjustment parameters and final results.
The issue of poor stability of internal orientation of cameras was described by Cramer and co-authors [Cramer 2017]. The research has showed out that in some cameras the focal length and principal point position are changing in several dozen of microns, although between calibration the camera was not operationally used. It is worth underlining that also thermal conditions impact the parameters of interior orientation [Daakir, 2019].
The standard method of image orientation, called indirect georeferencing, is a rigorous solution by bundle adjustment using Ground Control Points. The need to measure GCPs significantly increases the work time and the cost of UAVbased mapping. The fundamental question is about the minimal number of GCPs and its localization in elaborated area which are need to achieve a good precision of image orientation. Most papers express the view that the GCPs number should be between 3 and dozen, and suggest 6 GCPs as an optimal number [James, 2017]. Several authors advise more than 2 GCPs per 100 photos [Sanz-Ablanedo, 2018]. Due to various characteristic of mapped area (size, shape, relief, land cover) there are no universal rules about optimal number of GCPs and their spatial distribution.
The direct georeferencing of UAV images has been developed for some years. Nowadays, many vendors install survey-grade RTK-GNSS receiver on board the UAV. The application of cameras pose with centimetre accuracy in SFM workflow opens the door to UAV elaboration without GCPs. There are some experiences that confirm the expectation [Gerke, Stöcker, 2019].
During the project, multi-variant flight over the test field was conducted. The flights were performed with a fixed-wing airframe equipped with PPK receiver on-board. Based on the conducted flight, many data sets have been prepared, which differ as follows: • flight configuration, • georeferencing method: (1) GCP + approximate EO, (2) GCP + precise EO, (3) precise EO without GCP, • additional correction model for camera calibration, • key point/tie points limit: (1) without tie points, (2) 10k/500, (3) 40k/4k, • tie point filtering, • a priori accuracy settings.
The main aim of the research was to create database based on the conducted flight. The database containing hundreds of independent adjustment variants which allowed to investigate the separate influence of each factor on to final results. Based on the obtain database multiple statistical analysis were conducted to investigate: • Influence of using Fourier additional correction model, • Influence of georeferencing method on correlation between interior and exterior orientation • Influence of tie points count and distribution • Influence of observation weighting

TEST FIELD
Test field used for the investigation was located in residential area in Bochnia city covered area of average 350 m x 400 m. The size of the test field enable multi-variant coverage within one flight with fixed wing UAV. The area of the test field is characterized by a large number of internal roads, several storey blocks and terrain altitude differences up to 35 meters. There was 70 check points distributed across entire area ( Figure 1). 11 points were marked and located on building roofs and 59 were natural marked ground points.
As a natural marked, points like parking lines and wells were adopted while as a roof points special targets made of PVC were used ( Figure 2). Check points were measured in three session from June 2018 to December 2018 using RTK and RTN method. Based on these measurements; the accuracy of the points on 2 cm horizontal and 3 cm vertical level was determined (

DATA CAPTURING
The flight were conducted using FlyTech UAV BIRDIE fixed wing equipped with on board one frequency PPK receiver in August 2019. (Figure 3). The flight consisted of four missions varying in GSD/flight altitude, and flight direction. What is important, between each mission there was no middle landing, which was possible thanks to long flight time and advanced mission planning options.
The flight was made with Sony RX1R II camera (Table 2) at respectively 155 m (1.8 -2.2 cm GSD) and 230 m (2.7 -3.2 cm GSD) above mean ground level.    To compare results of different georeferencing methods, UAV used were equipped with single-frequency PPK receiver Emlid Reach M+. In order to obtain the most accurate time synchronization between camera and the receiver they were connected via camera hot shoe. The PPK receiver was used only for determining image coordinates and not for UAV navigation.

CALCULATIONS
First step of the processing was to determine camera precise coordinates based on GNSS observation from PPK receiver. This calculation was made in open source RTKLib software in reference to local Base Station and regional CORS (36 km away from test field). For variants with indirect georeferenced (NAV) approximated camera coordinates were obtained from standard code-based receiver used for UAV navigation.

Figure 4. Flight trajectory processed with PPK
All photogrammetry processing was performed with Agisoft Metashape Pro 1.6.2.
As the project aimed to perform multifactorial adjustments analysis it was important to minimalize human factor between each variant. To meet that requirement first step was to prepare project and perform all check points marking on images in chunks with independent flight configurations (variants 1-4). That prepared chunks were respectively merged to create variants 5-9. This allowed to minimize the impact of marking inaccuracy between calculation variants. Than project was copied to create 4 independent project divided by: • georeferenced method o PPK -with precise camera coordinates o NAV -with approximate camera coordinates • key point / tie point limit in image alignment setting : o 40 000 / 4 000 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B5-2020, 2020 XXIV ISPRS Congress (2020 edition) o 10 000 / 500 Than image alignment process for each project were conducted and saved as a basis for further analysis. Image alignment process were done with High accuracy setting (pixel level matching) and Reference and Generic preselection.
Based on that prepared projects, adjustments were made, which took into account the following factors: Finally, 64 independent adjustment scheme resulting from the combination of presented factors, were made. While, all of this scheme include 9 variants of flight configuration and 2 variants with/without additional camera calibration corrections, we obtained 1152 adjustment results which were stored in the database. Detailed information on the parameters for individual schemes is provided in the Table 7.
It is worth to highlight that in variants with precise camera coordinates (PPK) and without Ground Control Points marker image measurements accuracy were set to 1000000 pixels. As check points are treated as tie points that setting allowed to eliminate their influence on the adjustment.
Further calculations and analysis was performed in Statistica software.

DATABASE
All results of the adjustments were stored in Excel database which include: • mean, standard deviation and RMSE errors of estimated external orientation referred to PPK observation   Before proceeding with detailed analyzes of the influance of individual factors, the normality of the variables distribution were checked. At this point, it should be noted that all variables describing the results as RMSE errors and disparity (min-max) are characterized by a lognormal distribution due to the adoption of positive values. Therefore, at the beginning Box-Cox transformation was performed for these variables, which allowed normalization of their distributions and performance of multivariate parametric ANOVA tests.

Figure 5. Box-Cox transformation
For most analyzes, checkpoint RMSE 3D errors were used as measures of absolute accuracy and DISP MAX errors as measures of relative accuracy. At the same time, it should be noted that in projects with a high density of checkpoints, the mean square errors may be lower than the actual errors, therefore it is important to check whether the relative accuracy and if maximum errors do not exceed the acceptable thresholds.
The first order of one-way ANOVA was performed to determine whether the individual factors have a significant influence on the results obtained. While the rest of the article will be described multiple factor analysis examining the interaction between factors.
The first factor given to the analysis was checking the significance of using additional calibration parameters on the results. The one-way ANOVA test showed a clear advantage. The use of additional calibration parameters has significantly reduced the absolute RMSE error and disparity errors. The use of additional calibration parameters reduced the average RMSE errors by 17.5% and disparity errors by 19% while maintaining a significant difference in 95% confidence intervals.
Another factor that was analyzed was the impact of the method of obtaining images coordinates. One-way analysis showed no significant differences in the RMSE error with a significant reduction in disparity error for studies based on data obtained from the PPK receiver. The reduction was 18% while maintaining a significant difference in confidence intervals.

Figure 7. One-way ANOVA -source of camera coordinates
Next, the impact of the number of GCPs was analyzed. The analysis was made divided into PPK and NAV variants. As expected, the analysis showed that as the number of points increases, RMSE errors and disparity are significantly reduced.

Figure 8. One way ANOVA -number of GCP
In the case of analyzing the impact of a priori image coordinates accuracy, the range of analyzed variants was limited to PPK variants. The analysis carried out in this way The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B5-2020, 2020 XXIV ISPRS Congress (2020 edition) showed a significant reduction of RMSE error (15%) while reducing assumed a priori accuracy. At the same time, the disparity error in both variants did not show significant differences.

Figure 9. One-way ANOVA -a priori camera coordinates accuracy
The next analyzed factors were the influence of a priori GCP accuracy. The results of the analysis indicated a particularly significant impact of excessive a priori errors related to the measurement in images. In turn, the proposed thresholds for terrain accuracy of points did not show a significant impact on the obtained results.

Figure 10. One-way ANOVA -a priori in image measurement accuracy
The analysis of the impact of the tie points determined in the image alignment process showed a small, however significant reduction in the case of an increased limit. Figure 11. One-way ANOVA -tie point limit The analysis of the impact of filtration and the a priori accuracy of the tie points showed no statistically significant differences.
Next, the impact of cross-flight and with altitude change flight configuration was checked. Both of these factors have shown a significant impact on the reduction of resulting accuracy.

Figure 12. One-way ANOVA -flight configuration
The next stage of the study was to perform multiple factor ANOVA analysis to answer the question whether there are significant interactions between individual factors. At the same time, it should be noted that Statistica software allows to simultaneously check the relationship between a maximum of 4 factors.
First, the relationship between additional calibration parameters, source of image coordinates and limit of tie points was checked. The ANOVA test showed that the positive error reduction resulting from the application of additional calibration parameters is independent of the other factors. And Tukey's post hoc test confirmed this relationship for all individual averages. At the same time, the intergroup factor variance test showed a relationship between the source of the image coordinates and the limit of tie points. Although a reduction in errors was noted for all variants as the limit increased, a detailed post hoc test indicated that the significant reduction only applies to disparity errors for NAV variants (p value = 0.047824).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B5-2020, 2020 XXIV ISPRS Congress (2020 edition) Figure 13 ANOVA. -additional parameters for camera calibration, source of camera coordinates and tie point limit Then, for the PPK variants, the relationship between a priori accuracy of images coordinates, the limit of tie points and a priori accuracy of tie points was checked. The test showed that there is a relationship between all these factors. The relationship between the tie points limit and their assumed accuracy turned out to be the most important. The results show the inverse relationship. However, taking into account all factors, it has been shown that the impact of a priori accuracy of photo coordinates becomes significant as the limit of tie points increases. The shown relationships have also been confirmed in the case of division into groups due to the use of additional calibration parameters. Another analysis checking the relationship between the additional calibration parameters, image coordinate source and the cross flight configuration showed that the significance of the impact of the additional parameters depends on the cross flight configuration. The test showed that in the case of crossoverlay, the reduction of errors due to the use of additional calibration parameters is significantly greater than in the case of unidirectional flight. At the same time, the test showed a relationship between the source of the photo coordinates and the cross configuration. Thus, we can conclude that the benefits of cross flight configuration are particularly relevant for NAV variants.

Figure 15. ANOVA -cross flight configuration, additional parameters for camera calibration and source of camera coordinates
Further analyzes also showed the relationship between a priori accuracy of field and in image measurement of GCP and the number of GCP. The obtained results showed a significant impact of a priori in image measurement error values. At the same time, it has been shown that there is a relationship between both a priori errors. The indicated relationships have been confirmed regardless of the number of GCP and the use of the calibration model with additional parameters.

Figure 16. ANOVA -number of GCP, a priori in image measurements accuracy and a priori GCP accuracy
As part of the analyzes, influence of factors such as the source of image coordinates, the use of additional calibration parameters and the flight configuration on camera calibration The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B5-2020, 2020 XXIV ISPRS Congress (2020 edition) stability were also checked. Because all flights were made without middle landing, it was expected to obtain repeatable calibration results. The results obtained for the focal length showed above all that the use of additional calibration parameters causes a significant change in the calibration parameters and, at the same time, greater instability in their determination. Greater calibration instability was also demonstrated for NAV variants relative to PPK.

CONCLUSIONS
The ANOVA tests made based on the results obtained from multi-variant adjustments allowed to determine the effect of individual factors and their interaction on the final accuracies.
The advantages of using the Fourier model with additional calibration parameters have been confirmed. The obtained results showed a significant impact of the model on improving the obtained accuracy. Independence of the model's influence from such factors as the georeferencing method, the number of GCP, or the limit of tie points has been demonstrated. Thus, they allow concluding that the Fourier model implemented in Agisoft software, despite the use of as many as 96 parameters, does not show a problem related to overestimation and can be used with advantage in UAV photogrammetry.
Obtained results confirmed the advantages of using a precise camera position. Using the direct georeferencing method allowed obtaining centimeter-level accuracy even without Ground Control Points. It also allowed minimizing relative accuracy in all cases. Furthermore, due to the greater stability of determining camera calibration parameters, it can be stated that the use of precise image coordinates reduces the problem related to the correlation between external and internal orientation parameters.
It has been shown that the alignment parameters such as the in image measurement accuracy or a priori image coordinates accuracy have a significant impact on the obtained accuracy. At the same time, it has been shown that some of the parameters show intergroup relationships.
No significant impact of the factors such as tie point filtration, tie point limit, or tie point a priori accuracy show that the alignment algorithms are good at filtering incorrectly determined tie points. Thus, increasing the number of binding points has a positive effect on the accuracy of the determining elements of internal and external orientation.
Number  The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B5-2020, 2020