OPTIMIZATION OF GROUND CONTROL POINT ( GCP ) CONFIGURATION FOR UNMANNED AERIAL VEHICLE ( UAV ) SURVEY USING STRUCTURE FROM MOTION ( SFM )

This research presents a method in assessing the impact of Ground Control Point (GCP) distribution, quantity, and inter-GCP distances on the output Digital Elevation Model (DEM) by utilizing SfM and GIS. The study was carried out in a quarry site to assess the impacts of these parameters on the accuracy of accurate volumetric measurements UAV derivatives. Based on GCP Root Mean Square Error (RMSE) and surface checkpoint error (SCE), results showed that the best configuration is the evenly distributed GCP set (1.58 m average RMSE, 1.30 m average SCE). Configurations clumped to edge and distributed to edge follow suit with respective RMSE (SCE) of 2.53 m (2.13 m) and 3.11 m (2.54 m). The clumped to center configuration yielded 6.23 m RMSE and 4.66 m SCE. As the number of GCPs used increase, the RMSE and SCE are observed to decrease consistently for all configurations. Further iteration of the best configuration showed that from RMSE of 4.11 m when 4 GCPs are used, there is a drastic decrease to 0.86 m once 10 GCPs are used. From that quantity, only centimeter differences can be observed until the full set of 24 GCPs have been used with a 0.012 m error. This is reflected in the stockpile measurement when the iteration results are compared to the reference data. The dataset processed with a minimum of 4 GCPs have a 606,991.43 m difference, whereas the dataset processed with 23 out of 24 has a 791.12 m difference from the reference data. The accuracy of the SfM-based DEM increases with the quantity of the GCPs used with an even distribution.


INTRODUCTION 1.1 Background of Study
With the growing use of Unmanned Aerial Vehicle (UAV) in the past few years, it is undeniable that its application has significantly transformed from defense-focused applications to industrial and environmental applications due to its capability to work on remote and complex environments ease of deployment, and reasonable cost (Anderson and Gaston, 2013).
In comparison to the traditional aerial photogrammetry, UAVs prove to have leverage in terms of operating cost, efficiency of conducting flights, portability of the platform, and integration of different types of sensors.
Currently, this technology is utilized in automated mapping in surveying industry for time-efficient operations ( Remondino et al.,2016), research on disaster management (Adams and Friedland, 2011) and damage assessment ( Achille et al., 2015) , and environmental change detection such as for natural landform mapping (Harwin, 2015), quantifying volume change in glacier for different seasons (Gindraux et al., 2017), and crop monitoring (Honkavaara et al., 2012).The expanding demand for various applications clearly calls for further studies in exploring ways to get more reliable information from the data derivatives.In order to attain this, a method in obtaining accurate imagery and generating data outputs should be further looked into.A number of studies have been conducted in determining different ways on how to improve the accuracy of the outputs generated from UAV imagery: from the environmental conditions and flight settings during image acquisition (Ravzynski, 2017), attempting a minimum distance approach for ground control points (Pourali et al., 2014) and different spatial patterns of them (Wang et al., 2012) (Ridolfi, et al., 2017), evaluation of various positioning systems (Ruiz et al., 2013), to the processing parameters such as feature extraction, image matching, and bundle adjustment (James et al., 2017), up to the algorithm of creation of dense point cloud (Rosnell et al.,, 2012) and Digital Elevation Model DEM) (Ruiz et al., 2013) and polynomial transformation for image rectification using GCPs (Muhaisen, 2016), One of the factors that can significantly improve the quality of the data products is the use of accurate and well-distributed Ground Control Points (GCPs) to tie down the model properly to the ground values.To ensure both global and internal accuracy across datasets in different period, the location, distribution, and number of GCPs should be considered in establishing them in the area of interest.Since the use of GCPs translates to time and money in an entire survey operation, one way of ensuring the production of accurate and consistent data alongside minimizing operation time and cost is to determine the optimal way of GCP establishment.This research primarily aims to optimize the GCP configuration to ensure accuracy and processing efficiency when conducting a UAV survey.Particularly, it focuses on two things: assessing the impact of the GCP distribution, quantity, and inter-distance with the accuracy of the output DEM and how accurate and consistent stockpile measurements can be carried out from the output terrain.

Materials and Tools
The UAV used to acquire images is DJI Phantom 4 Pro, with a focal length of 8.8 mm, ISO 100, shutter speed of 1/2000, sensor width of 13.2 mm, sensor length of 8 mm, image size of 5472 x 3648 pixels.The GCPs were measured using a GNSS receiver and post-processed using a standard and precise GNSS positioning package.The aerial survey was conducted in a quarry site as a case study to assess the accuracy of volumetric computation.Twenty-five GCPs are initially recorded.However, from the post-processing results, one GCP yielded a variance exceeding the acceptable range of not more than 15 mm in the horizontal and not more than 50 mm in the vertical.This was tagged as a bad-fix data and was not used in the image processing.
Shown in Figure 1 is the lay-out of the full set of GCPs in the area.

Structure-from-Motion (SfM) Processing
The application of Structure-from-Motion method in generating DEM based on stitching of UAV imagery has been recently carried out, along with Multi-View Stereo (MVS) algorithm (James et al., 2017, Harwin, 2015).Compared to the traditional aerotriangulation, the automated workflow in feature extraction and bundle adjustment can be significantly more efficient in terms of time and resources specially when processing large datasets.This study made use of a SfM-MVS platform (Agisoft Photoscan V1.4.1.) in the creation of SfM-based DEM from the acquired UAV imagery.The workflow for the SfM processing is shown in Figure 2.

Reference Data
Before proceeding with the error propagation for the parameter assessment, a reference data is initially processed with the complete set of GCPs.This will be considered as the "truth" data (i.e., most accurate) to which all of the simulation configuration outputs will be compared against.The initial assumption is that the more well-distributed GCPs there are, the more accurate the output model is.It is only logical to use the dataset processed with the full set of 24 GCPs as the reference source.
A preliminary parameter setting test was conducted to determine the most appropriate SfM settings and parameter values.This includes the accuracy setting, preselection, keypoints and tiepoints limit, filter, and DEM resolution.The accuracy settings determine the estimate of the camera position.The higher the setting, the greater the software will upscale the images but at a cost of a longer processing time.This study chose Low setting to make up for a balance between accuracy and time efficiency.Generic and reference preselection are both enabled to thoroughly match overlapping photos for feature detection.Whereas the keypoints and tiepoints number limit dictate the maximum number of feature points to be extracted for each of these images.The default values are 40, 000 and 4, 000 which are also applied in the processing.There is no filter applied in the study as there is a tendency that stockpiles will be affected when outliers are attempted to clean up.From the result of the parameter setting test, unfiltered setting also yields the least surface checkpoint error.The objective in choosing best suited setting and values is the balance in less computational time without compromising the accuracy of the data.The same SfM parameter setting and values are applied all throughout the simulation configurations for the three error propagation test, keeping them constant while modifying the rest of the parameters such as GCP configuration.In order to assess thoroughly the behaviour of the GCP distribution, quantity, and inter-distances, a more thorough error propagation tests were conducted as explained in the following sections.

Error Propagation I: Distribution Test
For the first error propagation, four main distribution categories are identified: A) Clumped -center, B) Distributed -edge, C) Clumped -edge, and D) Distributed -edge and center.Each of the four configuration classes are processed with increasing GCPs, while keeping the rest of the unused control points as check points.To sum up, there are twenty-four simulation configurations, with six dataset iterations for each configuration class (Figure 3). (1) (2) where ∆  = residual of ith value in X axis ∆  = residual of ith value in Y axis ∆  = residual of ith value in Y axis n = number of control point Surface checkpoint error is the second accuracy parameter considered in this test.In order to do this, the DEM for each simulation is generated and imported in a Geographic Information System (GIS) platform (QGIS 2.18.18 'Las Palmas').
More specifically, the Point Sampling Tool is used to extract the elevation values from the DEM GeoTIFF file using the polygon attributes from the GCP shapefile.This will pull the elevation values of the same xy location of the GCP vector points.To get the checkpoint error, the extracted value is treated as the predicted value and the GCP elevation value is therefore treated as the true value where Similar to the GCP RMSE, the resulting surface checkpoint error for each simulation configuration are plotted against the GCP quantity and compared with each configuration class.
The aim of this test is to determine which configuration distribution would yield the least error from the two accuracy criteria, GCP RMSE and surface checkpoint error.The result is then propagated more robustly in the following error propagation test for GCP quantity assessment.

Error Propagation II: Quantity Test
The quantity test is a repetition of the distribution test processing workflow.However in this case, the result of best configuration class is further propagated by processing the dataset with one GCP increment from a minimum of four control points.This is done to understand more thoroughly the behaviour of the GCP when established using the most optimal distribution.
Using the GCP RMSE and surface checkpoint error as accuracy criteria similar to the first part of error propagation, the values are again tabulated and plotted against the GCP quantity to assess the trend.
While some studies have explored on computing the optimal GCP density (Gindraux et al., 2017), this current research emphasizes the assessment method to determine the performance of increasing GCP quantity and its effect on the accuracy of the terrain.This workflow is highly directed on a more strategic planning from a logical decision in monitoring a specific site over time to ensure data consistency.

Error Propagation III: Distance Test
To have a comprehensive analysis on how each of the control point affects the accuracy of the SfM-based elevation model, the Leave-One-Out (LOO) method is employed.This works by excluding one GCP from the dataset processing and model generation.Since there are a total of twenty-four control points, there are also twenty-four resulting LOO configurations.By computing the two accuracy parameters as employed in the first two error propagation tests, GCP RMSE and surface checkpoint error, the trend can then be plotted and the maximum and minimum LOO configuration can be easily identified.
With this information on hand, we can then further analyze whether or not the inter-distances of GCP affect the accuracy of the output terrain.
A distance matrix is created using a tool in the GIS platform.
From the vector point layer of the GCPs, this is used both as an input and target layer from which the distance is computed.This will generate a DataBase File (.dbf) which contains the corresponding distance computed from and to each control point.
Looking specifically at the group with the maximum and minimum error output, their distance matrices are then separated and further assessed.

Measurement Validation
One of the main objectives of this study is to carry out accurate stockpile measurements by optimizing the configuration of the control points.From the three error propagation tests, the best configuration class is further analysed by generating a SfMbased DEM for each simulation configuration and obtaining a stockpile measurement from it using spatial tools.The volume comparison is computed in reference to the reference data processed with the full set of control points.
The cut and fill volume measured for each configuration dataset is plotted for all iterations to be able to visually assess the differences more clearly.Results provide more insights on how the quantity of GCPs and their placement can affect in accurately measuring stockpiles in site which will provide a more scientific approach in quarry site management and survey planning.
Raster elevation file comparison is also produced and evaluated to have a spatial understanding on how the distribution and quantity of GCPs affect the terrain values generated from the SfM platform.Using the Reference data as the source of truth and the rest of the iterations as the datasets for comparison, spatial raster calculator is used to get the difference between the two elevation models.This is done for all the iterations from the predetermined best configuration class from the initial error propagation test.

Error Propagation I: Distribution Test
Two accuracy criteria are evaluated for each simulation configurations for the four identified distribution class.The resulting values are then plotted with respect to the number of control points used for each iteration.

GCP RMSE
Comprehensible graphs are created for both horizontal (xy) and vertical (z) axis from the output GCP RMSE for each configuration.This is to assess further the impact of the corresponding axis to the computed total magnitude RMSE.It is noticeable that the error values of the vertical axis have are significantly larger compared to the values in the horizontal axis.The dominance is apparent when the total magnitude plot is inspected further.
Generally, the values of the GCP RMSE decrease as the quantity of the GCPs used in the SfM processing increases.This can be consistently observed for all configuration class.Similar behaviour is seen both in the horizontal and vertical axis, which is conceivably, reflected in the result of the magnitude GCP RMSE.
A deeper look into each of the configuration class leads to the conclusion that configuration A consistently yields the highest error, whereas configuration D shows the least error values amongst the four configuration classes.It can be recalled that configuration A has a clumped distribution while configuration D is the even-area distribution.

Surface Checkpoint Error
To check if this is also the case with the surface values, checkpoints are obtained as the second accuracy criteria.Similar to the GCP RMSE, values are also plotted against the GCP quantity.
As can be seen in the graph, the resulting trend of the surface checkpoint error follows the trend of the GCP RMSE plotted results.This confirms and validates the finding that the increase in quantity of the GCPs, combined with an even distribution of the points in the area will yield the least error in the data.

Error Propagation II: Quantity Test
A more robust SfM processing iteration is conducted in the second part of the error propagation to further evaluate the behaviour of the GCP quantity when it is increased with one control point increment.The configuration class used for this experiment is class D -the configuration which yields the least error from the distribution test.

GCP RMSE
Similar to the previous analysis, the horizontal and vertical axis are plotted separately to visualize how they impact the over-all RMSE magnitude.The error in the z axis proves to be constantly producing a higher error in comparison to the horizontal error.Its dominance is evident when the magnitude has been computed.From a minimum of four GCPs used until nine, there is a drastic decline of RMSE values.Logically, the even distribution of the control points also greatly helped in reducing the error.As the more even the points are established in the area, the better the distribution of the error is expected.

Surface Checkpoint Error
The surface checkpoint error is obtained in the similar fashion as the previous analysis.A comparable trend can be assessed visually from the graph of the error values and the GCP quantity.It is noticeable that as the GCP quantity used in the processing of datasets increase, there is a significant decrease in the surface checkpoint error values.This observation is consistent with the initial iteration results.From the graph, it can be noted that there is an apparent stability of the error when the count of GCP reaches ten.Coming from the minimal four used GCPs, there is a large rate of change in the error until the tenth GCP is used.From here, there is a noticeable minimal change in the error until the full set of GCPs are used.

Error Propagation III: Distance Test
With the general SfM processing workflow as a reference method, the third and last error propagation test is conducted in a different set of simulation configurations.By employing LOO method, each of the GCP is taken out and results are assessed to determine how this would affect the accuracy of the resulting terrain.
From this, the configuration which yields the maximum and minimum error can be identified for further analysis.Results show that these are LOO 6, LOO 15, and LOO 21 for the maximum error group.LOO 9, LOO 10, and LOO 14 is included in the minimum error group.Notice the value of distances between GCPs.For the high error group, the GCP

Measurement Validation
Taking the largest stockpile in the quarry site, a volume measurement is carried out for all the twenty-four simulation configurations from the second error propagation.The results are compared against the reference data.

CONCLUSION AND FUTURE WORK
This research focuses mainly in the creation of a workflow in assessing the impact of the distribution, quantity, and interdistances in the area for quarry site management and monitoring.
The distribution test showed that amongst the four major GCP configurations, there is a general trend of a decrease in the incurred error for GCP RMSE and surface checkpoint error computation when the control points used in the processing are increased.When the x-y and z error values are examined separately, it can be noticed that the error in the vertical axis is significantly higher as compared to the horizontal error.This is apparent when the magnitude error is calculated for all iterations.For x-y axis, the GCP RMSE trend for all configurations inconsistently goes up and down when the quantity of ground control points is increased.But noticeably, when the CP RMSE is checked, all configurations would yield a decrease in x-y error as more GCPs are used.Similar observation can be seen in the Looking at both values of these two accuracy criteria, the most favoured distribution would be Configuration Class D, as it consistently incurs the lowest GCP RMSE and surface checkpoint error among the other configuration classes.
Whereas from the graph, Configuration Class A has consistently the highest GCP RMSE and surface checkpoint error, both for the horizontal (x-y) and the elevation values (z).From here it can be concluded that the more distributed your GCPs are placed in an area, the less error it will incur and the more concentrated the GCP placement is, the higher the error of the terrain will be.
To further assess the effect of the GCP quantity using the two accuracy parameters, Configuration Class A is further analyzed by processing with 1 GCP increment.Based on the results, it can be observed that there is an inversely proportional relationship between the GCPs used and the GCP RMSE incurred.As we increase the quantity of the GCPs, the GCP RMSE decreases.Once the GCPs used reach up to 10, there is a noticeable stability of the error values until it reaches the acceptable 2 cm error threshold with 23 GCPs.The similar behaviour can be observed in the surface checkpoint error.
In examining the effect of GCP inter-distances, LOO method is implemented.This is to determine how each point behaves and impacts the overall accuracy of the model.By getting the configurations with the highest and lowest incurred values of the accuracy criteria, their distance matrices are computed for further statistical analysis and comparison.Results show that the high error group gcp inter-distances are almost half the distance of the low error group.This supports the initial findings that GCPs should be placed with even distribution in the area.
All in all, the utilization of SfM and GIS platform in the GCP optimization can be a useful tool to ensure data consistency.By determining the best parameter setting, distribution configuration, optimal GCP, and GCP inter-distance, internal and global accuracy can be achieved.It is important to note that ultimately, the distribution, quantity, and distance should be taken in full consideration as the combination of these parameters will positively improve the accuracy of the resulting SfM-derived outputs.

Figure 2 .
Figure 2. General workflow for the SfM processing

.Figure 3 .
Figure 3. Configuration classes defined for distribution test.The empty triangles denote the checkpoints while the shaded triangles represent the points used for ground control

Figure 5 .Figure 6 .
Figure 5.Total GCP RMSE of the four configuration class in Z axis with respect to the GCP Quantity Figure 8.Total GCP RMSE in X-Y axis plotted with respect to the GCP Quantity

Figure10Figure 11 .
Figure10.Total GCP RMSE magnitude plotted with respect to the GCP Quantity

Table 1 .
SfM parameter settings in Agisoft Photoscan The final selected SfM parameters are tabulated in Table1.

Table 2 .
are actually large.Although the nearest GCP is as small as 2 m distance, this can vary as high as 18 m.For the low error group, most of the GCP inter-distances are actually low.This means the location of the control points are near to almost all the other surrounding GCPs Statistical analysis of the distance matrix for the maximum and minimum group