INITIAL STUDY ASSESSING THE SUITABILITY OF DRONES WITH LOW-COST GNSS AND IMU FOR MAPPING OVER FEATURELESS TERRAIN USING DIRECT GEOREFERENCING

: Drone technology has shown the potential to act as the middle ground between satellite, light aircraft, and terrestrial or in-situ methods. However, featureless terrain such as water poses a challenge when it comes to drone mapping. The main challenge is identifying matching points to combine overlapping images into a single dataset. In particular, because traditional methods such as Structure from Motion (SfM) is dependent on tie point collection, its usage over featureless terrain is almost impossible. In solving this problem, we propose that the use of Direct Georeferencing (DG) in registering images be explored as a potential method and we propose a method for correcting errors due to tilt with low-cost IMUs. This study first assesses the accuracy of direct georeferencing using low-cost Inertial Measurement Units (IMU) and Global Navigational Satellite System (GNSS) providing analysis of the error sources associated with direct georeferencing and then demonstrates new approaches to minimize them. To best simulate a water type environment or surface for the initial studies, a drone survey was conducted on flat farmland and a POSE analysis was performed. We then processed the images using direct georeferencing and then compared our error minimisation method to standard Bundle Block Adjustment with GCPs and again with no GCPs. Results showed that using the method proposed in this study helped reduce the Mean Absolute Error associated with direct georeferencing by 54%. These initial results show a clear potential for mapping over inland water using direct georeferencing.


INTRODUCTION
Featureless terrain can be described as a terrain with the absence of ground features. For example, water can be described as featureless terrain and one that is most difficult to map in photogrammetry due to its dynamic nature and homogenous appearance (Knaeps et al., 2019). Mapping featureless terrain is challenging due to the absence of or limited amount of ground features that can act as a tie point for surface reconstruction. As a result, tie point matching always fails in surface reconstruction for these environments. A recent study by Maravilla et al. (2019) which mapped the water column to estimate chlorophyll using a drone reported that the images that were captured over the water without shoreline were not calibrated using a classical bundle adjustment, as a result creating a hole in the orthoimage.
However, mapping water bodies for measurement of water quality parameters such as turbidity, Total Suspended Solids and chlorophyll-a are essential for good water management. The most widely used methods for water monitoring are in-situ measurements at sparse scales or medium resolution satellite imagery (Amanollahi et al., 2017;Chen and Quan, 2012;Hafeez et al., 2019;Isenstein and Park, 2014) to high-resolution satellite imagery (Du et al., 2018;Huovinen et al., 2019;Kupssinskü et al., 2020). However, studies have revealed that these approaches have many limitations. The in-situ measurement approach is time-consuming (Gholizadeh et al., 2016), labour intensive (Dlamini et al., 2016), costly (Keller et al., 2018), cannot be performed frequently and importantly the required spatial coverage cannot be achieved (Gholizadeh et al., 2016). Satellite images struggle to provide the level of detail needed for small scale water pollution assessment (Wu et al., 2019). The spatial resolution of satellite images does not provide the finer detail that the drone dataset provides. The finer details are essential for detecting patterns of water pollution. Moreover, with increased temporal resolution ( several times each day) from drones, a rapid response to water pollution outbreaks is also possible (Wu et al., 2019) and this facilitate an understanding of the stages of the outbreaks (Kubiak et al., 2016). Furthermore, due to low temporal frequencies of satellite images, they are often unavailable at required intervals. The user rarely has any control over the satellite (except when they have purchased satellite tasking) and they cannot develop an acquisition plan to reduce environmental impacts on image quality such as solar illumination effect. For example, Harmel et al. (2018) explained that the viewing geometry of the Sentinel-2 satellite makes it vulnerable to sun glint contamination, which is the specular reflection of light from the water surface which causes bright pixel values. The Sentinel-2 platform operates at a near-nadir viewing angle where sun glint is likely to occur. In some cases, sun glint can render an image over water unusable, causing the entire water surface to be characterized with bright pixel values.
Also, studies have shown that the probability of acquiring cloud-free data is influenced by region and month, further affecting its suitability (Corbane et al., 2020;Robinson et al., 2019;Wu et al., 2021). In Ireland in particular, frequent cloud cover over most parts of the year limits the use of satellite imagery (Dwyer, 2012). Moreover, the problem is exacerbated with satellites like Landsat 8 due to its longer revisit cycle of 16-days.
According to Cahalane et al. (2017), at a lower altitude, surveys from manned aircraft platforms such as aeroplanes and helicopters are very expensive, slow to mobilise and typically not applicable for regular surveys. The advance of drone technology has changed all of this and has shown potential in acting as a middle ground between satellite, manned aircraft, and terrestrial in-situ methods. Drones provide very high spatial resolution (1cm-3cm) datasets, operate below the cloud cover, and so provide cloud-free data in near real-time. Although it has proven potential for terrestrial applications, most obviously for topographic mapping (Suo et al., 2018;Syetiawan et al., 2020) and agriculture (Guan et al., 2019;Vanbrabant et al., 2019), issues concerned when mapping over water bodies are much less understood and these pose many challenges to enable the collection of accurate and reliable data. The main challenge is identifying matching points to combine overlapping images into an accurate orthomosaic.
Drone photogrammetry has benefitted from a fast automation process using structure-from-motion technology (SfM) which is a traditional and widely used method in computer vision for reconstructing a 3D surface by identifying identical points (called tie points) between overlapping images (Losè, Chiabrando and Tonolo, 2020;González-Jaramillo, Fries and Bendix, 2019;Vitti et al., 2019;Iglhaut et al., 2019). The SfM process starts by extracting and matching tie points from overlapping images using algorithms such as the scale-invariant feature transform (SIFT). When sufficient tie points are identified, a bundle block adjustment (BBA) is performed to calibrate the camera to produce the exterior orientation parameters and a sparse 3D point cloud of the surface. In this step, the user can completely rely on the Global Navigation Satellite System (GNSS) and the Inertial Measurement Unit (IMU) onboard or manually add Ground Control Points (GCPs) to improve the accuracy. The final step is to generate the Digital Surface Model (DSM) and the orthomosaic. However, because the SfM method is critically dependent on feature detection and matching, its usage over water is almost impossible. This is because water has a dynamic featureless surface, resulting in poor performance when using feature detectors such as SIFT, and also precludes the collection of manual tie points. In solving this problem, we propose that the use of Direct Georeferencing (DG) in stitching images over water can be a potential method. DG can be described as a method that measures the position and orientation of an airborne sensor and thereby directly determinates the exterior orientation parameters without any additional image-based measurements (eg. GCPs) (Ekaso et al., 2020;Lo et al., 2015). This method utilizes the drone's Global Position System (GPS) and onboard IMU. The GPS measures the latitude, longitude, and altitude, while the IMU sensor captures the angular values for Roll, Pitch, and Yaw of the drone. The GPS and the IMU data are used to transform the raw image into an accurate orthogonal representation of the ground thereby making it possible for the measurement of distance, positions and area. The major difference between the DG and SfM is that the former does not rely on tie points to create the orthomosaic.
Although there have been many successful applications of cameras from drones in mapping terrestrial environments, there is a clear gap in the knowledge of their suitability over inland waters. This study shows the (i) initial result on direct georeferencing using a low-cost IMU and GNSS and providing analysis of the associated error sources and (ii) demonstrates new approaches for correcting image displacement due to tilt.

Sensor and Platform
A Bluegrass VTOL (Vertical Take-Off Landing) drone manufactured by Parrot was used to acquire dataset imagery over the test site. The Bluegrass has a Skycontroller, a Sequoia multispectral sensor and a Sunshine sensor. The sunshine sensor records the ambient illumination from the sun and is used in calibrating the images. The Sequoia sensor has two important features: the IMU and GNSS, as can be seen in Figure 1. This helps in the precise geotagging of images. The IMU has an accuracy of 0.5 o for Pitch and Roll and 1.0 o for Yaw and a GNSS accuracy of ± 1.5 m for the horizontal and vertical plane (Sekrecka et al., 2020). The Sequoia camera is mounted beneath the Bluegrass at an angle of 15 o and is not mounted on a selflevelling gimbal.

Image Acquisition
For determining the performance of DG using the Sequoia camera for UAV-based digital orthomosaic generation over water, flat farmland was selected as a comparable test site. This test area was ideal as it allowed the placement of GCPs for accurate ground truth measurement and was selected as the best available approximation of the flat water surface. The variation in elevation was measured and identified as less than 1.6m. A drone survey was carried out using the Pix4dCapture appthis app enables autonomous flight by allowing drones to fly following a designated path/waypoint with the pilot supervising operations. A total of 6 flights lines were flown, which covered an area of 10.8ha, as shown in Figure 2. An overlap of 70% for both across and along track was specified and the flying height was set at 120m, which produced a Ground Sampling Distance (GSD) of 11.3 cm per pixel. The total number of images captured for the survey was 72 with 30 GCPs recorded using a Trimble 5800 GPS receiver with an accuracy of ±50mm. The GCPs were recorded in the Irish Transverse Mercator (ITM).

Processing of the Drone Data Using Pix4D
The initial 3D reconstruction to provide benchmark data of the surface was done using Pix4D software version 4.6.4. The image processing followed the usual workflow: image alignment and camera calibration, tie point extraction, bundle block adjustment (BBA), generation of dense point clouds, DSM and creation of orthomosaic (Franzini et al., 2019;Maravilla et al., 2019;Teppati Losè et al., 2021). Two orthomosaics were generated: an orthomosaic without GCPs and orthomosaic with GCPs. The orthomosaic with the GCPs was generated using 10 GCPs and the output was validated using the remaining 20 GCPs.

Direct Georeferencing
For reconstructing a 3D surface using photogrammetry methods, there is the need for a model which can project from a 3D scene to an image as shown in Figure 3.
The Yaw (ψ), Pitch (θ) and Roll (φ) transformation matrix is then determined by three consecutive rotation matrices in the following order: (1) Roll-rotation around the X-axis (2) Pitchrotation around the Y-axis (3) Yaw-rotation around the Z-axis. The combination of the three rotations create the following orthogonal transformation matrix: = Rz(ψ).Ry(θ).Rx(φ) In the context of photogrammetry Omega (ω), Phi (ϕ) and Kappa (κ) are required to transform a vector from an image coordinate system B to the object system E. The rotation transformation matrix is defined by counter-clockwise rotation angles ω, ϕ and κ: : = Rx(ω).Ry(ϕ).Rz(κ) The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France

Intrinsic Parameters:
The intrinsic parameters describe the interior orientation of the camera. They are parameters needed to model the geometry and the physics of the camera, namely; the focal length (f), principal points ( , ) and any parameter used to model lens distortion ( K1, K2, K3,P1, P2). The combination of the parameters will result in the following transformation matrix:

Processing of Images and Analysis
The image processing was done by implementing all the mathematical equations in Python. Various open-source libraries were used, for example, the images were projected to the ground plane using libraries such as the OpenCV and Micasense Python libraries. The data manipulation and analysis of the pose were carried out using Pandas and NumPy. The data was visualized and plotted using Matplotlib. The statistical analysis conducted included Mean Absolute Errors, residual analysis and standard deviation.

Image Displacement Correction
Despite IMU, gimbals, and other stabilizing equipment, in practice, it is impossible to maintain the optical axis of a drone camera truly vertical (Wolf et al., 2014) and images from lowcost drones without a self-stabilizing gimbal are most affected by tilt. In photogrammetry, different types of images can be captured during a survey, namely; nadir/near nadir, low tilt, high tilt and oblique imagery. When the camera axis is truly vertical or there is a small unintentional tilt which is less than 1 o , the image can be categorized as a nadir/near nadir image (Wolf et al., 2014). For low tilt images, the tilt of the camera axis is equal to or not more than 3 o (Wolf et al., 2014). On the other hand, images captured with the camera axis unintentionally tilted more than 3 o can be considered as high tilt. According to Verykokou and Ioannidis, (2018), images that are captured with the camera axis intentionally inclined more than 5 o can be classified as oblique images. In 3D image reconstruction, the procedures suitable for analysing nadir/near nadir images can be used for low tilted images without serious consequences. However, this cannot be said for high tilted and oblique images. This is because, in such images, certain errors are introduced as a result of the tilt. For example, in a high tilted and oblique photograph taken at a certain height, the GNSS recording by the drone (GP) and the Ground Principal Point (GP') do not match as shown in Figure 4. This happens because when the camera's optical axis tilts away from the vertical, the Ground Principal Point changes. This causes a shift between the GNSS and the Ground Principal Point. This shift is primarily strong in images that have high tilt angles. In correcting the displacement due to tilt, this study proposes a formula that measures the shift in distance between GP and GP'on a flat terrain. In theory, a rotation in one axis affects the direction of the following axis. For example, a degree tilt in Roll will have a greater effect on the Y coordinates and a smaller effect on the X coordinate. While a degree change in Pitch will have a greater effect on the X coordinate and a smaller effect on the Y coordinate (Stam, 2010). This shift can be mathematically calculated for flat terrain using the law of Sine rule, given that we already know the flight height and the attitude from the IMU. The shift can be measured in X and Y using the equation (8) and (10). Subsequently, the image can be corrected using the shift values in X and Y.
= ℎ * tan( ℎ) (8) where bx is the shift value in X between GP and GP' by is the shift value in Y between GP and GP' bx1 and by1 is the X and Y coordinate for the shifted image h is the flying height and bxy is the shift distance between GP and GP' Figure 4. An illustration of a tilt angle. The image (left) shows an image taken when the drone is tilted at an angle which causes a mismatch between the ground principal point and the ground GNSS and vice-versa for the right image.

Planimetric Accuracy Assessment of Direct Georeferencing and Bundle Block Adjustment (BBA)
To analyse the effectiveness of the DG approach in mapping inland water, the accuracy of the approach must be measured against established photogrammetric methods such as BBA. To assess the planimetric accuracy of the Sequoia camera for BBA and DG approach, flat farmland was used as a test site to compare orthomosaics created using BBA with/without GCPs and DG. From the analysis, different accuracies were reported for the three approaches. The Mean Absolute Errors (MAE) calculated for each test were 0.14m,1.55m and 18.93m for BBA with GCPs, BBA with no GCPs and DG respectively, as shown in Figure 5. The result indicates that both the BBAs obtained acceptable accuracy for topographic mapping, however, as expected, the BBA with GCPs performed best. The DG approach recorded a high MAE and thus, the initial accuracy result would not result in a suitable orthomosaic. According to Yuan and Zhang (2008), in order to improve the accuracy of DG, the sources and effects of errors need to be recalled first.
The accuracy of the DG is determined by the GNSS and IMU, the stability of the platform and the accuracy of the system calibration (Costa and Mitishita, 2019;Grejner-Brzezinska, 2001;Mian et al., 2016). Hence, any error in the interior and exterior orientation parameters will directly affect the MAE of DG. This can be seen in Figure 5 where the high recorded MAEs directly coincide with periods of high Pitch angle. Figure 5. Analysis of Mean Absolute Errors (MAE) for positional accuracy using direct georeferencing before the proposed correction, BBA with GCPs and BBA with no GCPs

Analysis of the Sources of Errors Associated with Direct Georeferencing
The Sequoia camera is mounted beneath the Parrot Bluegrass at an angle of 15 o and is not mounted on a self-levelling gimbal. The forward motion of the Parrot Bluegrass means the drone tilts downwards. Due to the 15 o inclination of the camera, and as a feature of the drone to approximate a gimbal, the drone needs to pitch 15 o down for the Sequoia camera to be perfectly horizontal. Our test demonstrates that images captured at the beginning and end of each flightline contained high MAE and high mean pitch values of 33m and 19 o respectively. This is because, at the beginning and end of every flightline, the drone is stationary, thus hovering to take the first and last pictures. The Bluegrass then becomes almost perfectly horizontal and the Sequoia camera is then inclined at 15 o .
From the result, the mean angular ratings for the Pitch, Roll and Yaw were 6.7 o , 5.5 o and 88.2 o respectively. Also, the standard deviation was calculated at 6.4 o , 1.5 o , 1.9 o for Pitch, Roll and Yaw respectively. Adopting the definition of the types of images from Verykokou and Ioannidis, (2018) and Wolf et al., (2014), analysis of the 6 flight lines was done for the survey. The result revealed that 5.5% of the images were nadir/near nadir images, 41.6% were low tilted images and 52.9% were high tilted images. This implies that the majority of the images acquired contained tilt distortions and therefore require applications of our tilt correction methodology. Also, a recent study from Otsuka et al., (2018) explained that high wind speed and flight speed can increase the pitch angle of multirotor drones. Most importantly, it will influence the viewing angles of cameras that are not mounted on a self-levelling gimbal. In our study, it can be inferred that the wind speed had very little influence on the variations in the angular measurement. This is because the ground wind speed recorded during the survey did not exceed 1.6m/s, however, according to Otarola et al. (2019), wind speed increases by up to 0.25m/s per every 10m of altitude. Using the 0.25m/s factor, the wind speed calculated for the survey at 120m was 3m/s. This means that the wind speed was within the recommended range of <3.3m/s as suggested by Doukari et al. (2019). However, the flight speed (12m/s) could have had an impact on the IMU measurements. Figure 6. Attitude angles in Pitch, Roll and Yaw measured from the IMU

Direct Georeferencing Exterior Orientation Analysis Test
In order to evaluate the suitability of the attitude angles for DG, each attitude angle was plotted in a line graph in Figure 7. The refined Omega, Phi and Kappa from BBA with GCPs was used as the benchmark to compare the attitude angles of BBA without GCPs and DG. In order to do analytical photogrammetry, the Pitch, Roll and Yaw were converted to Omega, Phi and Kappa respectively. The result showed that the residual error for Omega (ω) angle for the BBA without GCP and direct georeferencing were 0.09 o and 1.38 o respectively. Also, the residual error for the Phi (ϕ) angle was 0.13 o and 0.77 o for BBA without GCP and direct georeferencing respectively. The highest recorded residual error was the Kappa (κ) angle which was 0.16 o and 4.91 o for BBA without GCP and direct georeferencing respectively. In the BBA approach, less accurate attitude measurement is of less importance because its effect is mitigated by overlapping images with sufficient tie points which improves the exterior orientation (Rehak and Skaloud, 2016). However, in DG, the accuracy of the measurement directly depends on factors such as sensor quality, wind speed and flight speed (Dreier et al., 2021;Jaud et al., 2018).

Improved DG Orthomosaic using Image Displacement Correction
The initial direct georeferencing errors were improved using the image displacement correction formula in section 2.6. From Figure 8, it can be seen that the DG MAE was improved from 18.93m to 9.34m. Also, there was a significant improvement in the standard deviation from the initial direct georeferencing result of 10.09m to 2.9m after image displacement correction. Our method helps reduce the DG MAE down by 53.54%, thus reducing the error by more than half. Additionally, our method helped smooth the peak errors and high tilt angles which were caused by the high pitch and roll values during the survey. It was shown that our method is suitable for correcting both low tilt and high tilt images. Comparing our result to the classical bundle block adjustment, it is clear that there are further errors that need to be removed to further improve the planimetric accuracy and this work is currently underway.

CONCLUSION
This paper showed the initial results of using direct georeferencing as a potential approach that can be applied in mapping over inland water. An empirical study was performed to assess the accuracy of direct georeferencing using a low-cost IMU and GNSS and providing analysis of the error sources associated with DG and then demonstrated new approach to minimize them. The outcome of the results showed that the Mean Absolute Error from the initial DG approach was reduced from 18.93m to 9.34m after tilt angle correction was applied. This means that the direct georeferencing approach can be improved by using the tilt angle correction method. This method also helped to smooth the high Mean Absolute Errors which result from high Pitch angles. The result outlined in this paper demonstrates the feasibility of using low-cost GPS and IMU for direct georeferencing over featureless terrain.
Inferring from these results, it is suggested that future research should focus on analysing the influence of flight parameters on orthophoto using direct georeferencing. Furthermore, future The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France research should focus on further refining the pose measurement and removing the tilt distortion over featureless terrain.

ACKNOWLEDGEMENT
This contribution was funded by John and Pat Hume-Maynooth University.