INFLUENCE OF CO-ALIGNMENT PROCEDURES ON THE CO-REGISTRATION ACCURACY OF MULTI-EPOCH SFM POINTS CLOUDS

The well-established spread of Remotely Piloted Aircraft Systems (RPAS) as high-performance devices in the acquisition of huge datasets has found a fertile field in the geomorphological change detection in coastal areas. The ability to retrieve image datasets with multi-epoch frequency makes them effectively incisive for planning ongoing monitoring. Considering the wide accessibility to multiple Structure-from-Motion (SfM)-3D point clouds, it follows the need for their proper management to identify a profitable coregistration approach valid for a proper comparison among them. In most cases the co-registration is inherited from the same georeferencing; in other cases, it can be done manually. Unfortunately, these methodologies are time consuming and often do not properly consider geometric errors on the models. The purpose of this research work was therefore to analyse an alternative method such as the co-alignment of sparse point clouds. Given the independently or co-aligned processed multi-epoch datasets, mean errors (ME) and root-mean-square error (RMSE) on Check Points (CPs) were evaluated by adopting different georeferencing strategies. Lastly, by first generating dense point clouds and from these the Digital Elevation Models (DEMs), scalar fields regarding DEM of Differences (DoD) were computed and allowed to localize any uncertainties δz among the estimated elevations. A cloud-to-cloud comparison was obtained using the M3C2 algorithm to extrapolate systematic georeferencing errors and the local deviation between models, an evidence of how the method can affect the detectable changes. The co-alignment methodology showed encouraging results proving to be a valid alternative to more traditional approaches.


INTRODUCTION
An accurate and high-resolution mapping of Earth surface is essential to describe its morphology and to understand the processes that rule the environment (Peppa et al., 2019;Capolupo et al., 2020b;Carvalho et al., 2020;Saponaro et al., 2020b). The baseline information of landform is provided by the Digital Elevation Model (DEM) extracted via different methods, i.e., for instance, remote sensing and conventional aerial photogrammetry (Tarantino, Figorito, 2011;Capolupo et al., 2018a). Nevertheless, over the years, these techniques have shown several limits in monitoring processes at plot scale (i.e., an area of sufficient homogeneity to be treated as a single unit), mainly due to high costs in terms of data acquisition and processing times. Therefore, collecting multi-epoch imageries using conventional techniques does not seem convenient at plot scale (Laporte-Fauret et al., 2019;Carrera-Hernández et al., 2020). This has encouraged the spread of alternative methodology, like Remotely Piloted Aircraft Systems (RPAS), appearing as high-performance devices in the acquisition of huge datasets in a short time and at low cost (Capolupo et al., 2018b;Lamsters et al., 2020). These small and smart aircrafts can be used to acquire imagery featured by Ground Sampling Distance (GSD) of centimeter-order and an unrestricted frequency of revisiting, except for any weather or regulatory limits (Ewertowski et al., 2019). These RPAS-based datasets can subsequently be processed using Structure-from-Motion (SfM) techniques to generate sparse point clouds. Subsequently these latter can be thickened with Dense Matching algorithms from which then orthomosaics, Digital Elevation Models (DEM) or detailed 3D reconstruction of the scene were achieved (Caprioli et al., 2007;Jiang et al., 2020;Meinen, Robinson, 2020). These are therefore generally identified as SfM-products, as the final characteristics are essentially related to how the sparse point clouds are processed. However, a functional change detection requires repeated surveys of the area of interest at the relevant geomorphic time scale, sufficient accuracy and precision to correctly interpret changes and their relevance (James et al., 2019). Moreover, a coherent co-registration among products for any accurate comparison becomes crucial (Saponaro et al., 2018;Saponaro et al., 2019a;de Haas et al., 2020). It is of key importance that the time-differentiated SfM-products are both accurate and spatially consistent. Hence, there is a growing need to establish a proper management of SfM-clouds starting from their generation up to the identification of a viable co-registration approach for a proper comparison among them (Coulter et al., 2019;Saponaro et al., 2020a). In a more traditional approach, co-registration is inherited from the georeferencing itself, or else it can be set manually by taking one cloud as a reference and overlaying the others in at least 4 point-benchmarks. Unfortunately, these methodologies are time-consuming and often do not adequately account for geometric errors on the models (Cucchiaro et al., 2020). The lack of a priori knowledge of optimal conditions makes difficult to identify a precise number of ground control points (GCPs), valid across different scenarios in shape and size. At most, on a comparable basis of scenario, technology and methodology adopted, the US Geological Survey (USGS) findings suggest that a number greater than 10 and with a high accuracy of less than 3 cm makes co-registration between models sufficiently accurate (Kasprak et al., 2019;Collins et al., 2009). In practice, the co-registration procedure was proceeded iteratively, reducing the registration root-mean-square error (RMSE) from time to time, significantly lengthening the operating time. Automated registration tools, such as CloudCompare's Iterative Closest Point algorithm (Peppa et al., 2019;Cucchiaro et al., 2020), were often used to automatically search and match thousands of points to reduce registration RMSE below a certain threshold. On the other hand, while coregistration can be considered as improved, errors can be exaggeratedly manipulated. The automated procedure generates distortions by exaggerating the fit of the clouds to the reference cloud, particularly in cases where the distribution of GCPs is not well dispersed. In short, this bias causes the automated registration to adjust the clouds to have a lower RMSE but at the same time gives them a false shape cause of erroneous change detection. More recently, (Feurer, Vinatier, 2018) first and (Cook, Dietze, 2019) later demonstrated that processing multi-epoch datasets as a single block in the alignment phase of SfM processing allows to compute coherent multi-temporal point clouds. Referred to as "Co-Alignment" (Cook, Dietze, 2019), this methodology requires aligning a much larger number of images, especially when combining multiple surveys. This certainly involves a non-linear increase in processing time and requires greater hardware capacity, but with an increase in computational capabilities aided by the strong parallelization of SfM methods (de Haas et al., 2020). (Cook, Dietze, 2019) demonstrated that the co-alignment of RPAS-derived images by direct georeferencing generates blocks for each dataset characterized by orientations comparable to those obtained by the classical approach by indirect georeferencing. In multi-epoch scenarios the individual epochs are processed and combined in a common framework, initially using the poses of the various images (Zhang et al., 2020). Subsequently, the blocks are optimized using GCPs, collimated in a single solution, or by direct georeferencing. Compared to the classical approach, co-alignment improves the accuracy of the detection of topographical changes by a factor of 4, in the case of indirect georeferencing, and by a factor of 3 with direct georeferencing (de Haas et al., 2020). Since this topic has only recently become an object of study, very little support literature is available. The aim of this research work is therefore to analyse the coalignment of sparse point clouds as an alternative method of coregistration. Given the multi-epoch datasets processed individually or co-aligned, mean errors (ME) and root-meansquare error (RMSE) on well-distributed Control Points (CPs) were evaluated by adopting different georeferencing strategies. Thereafter, running dense matching algorithms, from their outputs the Digital Elevation Models (DEMs) were computed. Output performance were assessed computing the DEM of Differences (DoD), capable of locating any discrepancies among the vertical estimations (Wheaton et al., 2010), and Cloud-to-cloud comparisons were obtained using the M3C2 algorithm (Lague et al., 2013), aimed at extrapolating the local deviation between models.

Pilot site and field activities
The rocky shoreline stretch of about 400 m, located in the southernmost district of Bari -Torre a Mare (Apulian region, Italy) (Figure 1), was selected as pilot site since affected by recurrent soil degradation phenomena, such as the collapse of cliffs. The area features a rugged coastline with typical stepped profile, characterized by an irregular elevation ranging between 1 and 5 m, with many small sandy bays and coves. Data acquisition campaigns were scheduled between 2018 and 2019 in compliance with national regulations. Specifically, RPAS photogrammetric pictures were collected in December 2018, March 2019, and October 2019. No flight missions were conducted between April and September 2019 due to the operational restrictions related to the beach season. Each campaign was performed adopting a DJI Inspire 1 Pro v.2 commercial quadcopter, equipped with a 3-axis gimbal fitted with a DJI ZenMuse X3 non-metric camera (focal length 3.61 mm, pixel size 1.56 μm, effective pixels 12.4M). Moreover, a low-cost GNSS/INS positioning receiver, set to record RPAS geographical coordinates in WGS84 (EPSG: 4326) reference system, and a barometer, aimed at storing the flight altitude, were used as well. The flight plan was programmed to obtain an average Ground Sampling Distance (GSD) of about 0.04 m/pix, a forward and side overlaps of 85% and 75%, respectively, by applying DJI Ground Station Pro application . This ensured to perform all field data campaigns using the same path, waypoints and flight conditions (e.g., cruising speed of 4.0 m/s and flight height of 100 m Above Ground Level (AGL)). Three datasets were built, one from each mission, consisting of 77 nadiral images. Georeferencing with 10 GCPs (marked red dots), Direct Georeferencing. CPs (yellow dots) in the three scenarios.
A GNSS survey campaign was also designed by identifying 30 permanent natural features easily detectable in all RPAS imagery ( Figure 1). Their position measurements were practiced with a Leica Viva CS10/GS10 GNSS receiver in Network Real Time Kinematic (NRTK) mode connected to the Leica SmartNet Italpos permanent station network. An average accuracy of 0.02 m on the measured targets was attested (Saponaro et al. 2019a). The system RDN2008/UTM zone 33N (NE) (EPSG: 6708) was adopted as reference.

Photogrammetric workflow
A processing workflow (Figure 2), consistent with the outcomes of previous works (Saponaro et al., 2020c), was planned to handle photogrammetric imagery, generating accurate point clouds and Digital Elevation Models (DEMs). It was run independently in three separate chunks (DEC, MAR, OCT), each of them created to process a specific dataset built from a corresponding flight mission. A last fourth chunk (CO) was also generated to treat the total of data acquired in all campaigns (231 pictures) simultaneously. The multi-chunks processing was performed in parallel in Agisoft Metashape (v.1.5.2) software on an Intel(R) Core (TM) i7-3970X CPU 3.50GHz workstation, featuring 16GB of RAM and an NVIDIA GeForce GTX 650 graphic cards. The Workspace Setting is aimed at ensuring the plausibility of each subsequent operation (Saponaro et al., 2020c). This step is mainly based on the definition of a reference system, needed to guarantee the consistency of orientation and scale of the final models, and camera calibration parameters, required to drastically reduce final error. Indeed, a lack of homogeneity of reference systems among acquired data and workspace encourages the propagation of errors in the final models (Sanz-Ablanedo et al. 2020). Without a purposed calibration of the parameters of the camera and its lever arm, errors might be even more significant. While a self-calibration procedure is commonly preferred, this can also be arranged through rigorous operations in laboratory Saponaro et al., 2020d). The Agisoft Metashape software uses Brown's 12component lens representation model (Fryer, Brown, 1986): in this work the radial and tangential distortion parameters above the third order (K3, K4, P3, P4, respectively) were not accounted in order to avoid overparameterisation issues (James et al., 2017). Lastly, as suggested by (Coulter et al., 2019), the illumination variation was balanced using Set Brightness option (Table 1) to equalize it across all datasets. Once the workspace was defined, the next step was Alignment. The software started the algorithms looking for key and tie points and their correspondences, image by image. The camera pose geometries were reconstructed: given the internal orientations obtained from the self-calibration estimates of the cameras and the positional information for each pose, a relative external orientation of the photogrammetric block was calculated (Eltner, Sofia, 2020). A sparse point cloud was thus returned. Generally, the sparse point cloud undergoes an ideal filtering treatment to reduce points characterized by a reprojection error above a threshold commonly set at 0.5 pixels. The photogrammetric block captures any corrections by compensating for estimates of relative image orientation quantified in the previous image alignment step. In Agisoft Metashape platform, selecting the Gradual Selection item sifts through several filtering options. Table 1 shows the filtering manipulations carried out on the sparse point clouds, according to (Saponaro et al., 2020c).
In the same step, georeferencing strategies were implemented. The coordinates of points measured in the field were loaded into the workspace and marked in all images, where distinctly visible. Thus, they were randomly split in GCPs and CPs without losing the homogeneity criteria in spatial distribution ( Figure 1) (Awasthi et al., 2020). The former group was useful for indirectly georeferencing (IG) the photogrammetric blocks; the latter for controlling the final accuracy. Three different setups, based on the adoption of i) 20 GCPs (red dots in Figure  1), ii) 10 GCPs (marked red dots in Figure 1), iii) 0 GCP (Direct georeferencing (DG) case (James et al., 2017;Padró et al., 2019), were carried out per each chunk. Selected CPs (10) were not modified in the three examined setups, as shown in Figure  1.
Going to the next step, adjustments to the estimates were enabled by means of the Bundle Block Adjustment (BBA) algorithms using the Optimize Cameras command. This command adjusts and refines the scene geometries and camera calibration (if authorized) while minimizing squared redesign errors between image and photogrammetric block points. The software aggregates heterogeneous information useful for BBA compensations: GCPs implemented in the block are fundamental for their accuracy (Saponaro et al. 2019c;Capolupo et al., 2020a). In the case of DG, the position information recorded in each image by the RPAS receiver were useful, even if they transferred less sizeable corrections due to their lower accuracy. The chosen parameterization for the performed computations is reported in Table 1, according to (Saponaro et al., 2020c).  Table 1. Summary sheet of the parameterization used in Agisoft Metashape processing. In Reference Settings, the parameterizations adopted are shown in order to make the workspace consistent with the instrumental limits and the image space of the software used. In Processed Planned, the choices made in the processes carried out. In particular, the labels DEC, MAR, OCT and CO indicate respectively the datasets of December, March, October and the one obtained by Co-Alignment. In Optimize Cameras only the camera selfcalibration parameters, belonging to the Model of Brown, not used in the BBA are reported.

GPS/INS Offset
Once the BBA phase was completed, the fourth chunk, related to the co-alignment case, was duplicated in three sub-chunks. This sub-chunk CO were subjected to a phase of split: precisely, in each of them were removed the images respectively referred to the acquisitions of (March, October), (December, October) and (December, March). Thus, the first sub-chunks involved just the pictures captured in December, the second one only the images collected in March and the third one just the photos collected in October. This operation does not affect the orientation of the blocks calculated in the previous step as the software maintains the estimates weighed during the Co-Alignment and BBA phase.
To evaluate the performance of generated models in all cases, Mean Error (ME), planar and 3D-dimensional Root-Mean-Square Error (RMSEP and RMSE3D), were estimated on CPs for the three georeferencing strategies implemented per each dataset. Thus, dense point clouds were generated and exported into .las format to be processed in CloudCompare software. The adopted parameters are reported in Table 1. Lastly, the corresponding DEM were generated too.

DEM of Difference Analysis
The DEM of Differences (DoD), estimated by subtracting the early epoch DEM (DEM1) from the subsequent period DEM (DEM2) (Equation 1), is a useful tool to evaluate geomorphological changes between multi-epoch surveys (Wheaton et al., 2010).

Cloud-to-Cloud Analysis
An appropriate test to examine any registration is related to the analysis of spatial trends in the data using a cloud-to-cloud comparison (Coulter et al., 2019). In this step, the Multiscale Model-to-Model Cloud Comparison (M3C2) implemented in the CloudCompare plug-in (Lague et al., 2013) was used to calculate the distribution of existing cloud-to-cloud distances between equal month and GCPs-implementation datasets. This provides an index of cloud-to-cloud variations. M3C2 results are affected by the radius (r) value selected to compute it. Therefore, identifying its optimal value for the scenario under investigation is essential to calculate these distances.  defined the criteria, based on the computation of the geometric characteristics of point clouds, such as, for instance, roughness and number of neighbors, to detect it. Therefore, such features were computed firstly by varying r from 0.05 to 0.50 m and identifying the best value of r able to descript the most significant geometric entity of the area (i.e., the rocky masses) (Di Francesco et al., 2020). The value of r was set equal to 0.35 m. Table 2 reports the characteristics of the generated point clouds, showing that, in terms of numerical entity, co-alignment procedures do not disturb the consistency of photogrammetric procedures. Given the inability to detect a precise systematicity among the datasets behavior and taking into account the considerations reported by (Coulter et al., 2019), it is evident that these low numerical variation ΔN is attributable to the light conditions at the moment of field data campaigns. Moreover, Table 2 allows to also detect the influence of georeferencing strategies on the consistency of the final point clouds, showing comparable results for the three setups. These findings are also

Photogrammetric outcomes
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B2-2021XXIV ISPRS Congress (2021 supported by the statistics (ME and RMSE) reported in Table 3. In all examined setups, ME and RMSE are in line with the standards defined by ASPRS for the digital production of cartographic data (American Society for Photogrammetry and Remote Sensing (ASPRS), 2015; Saponaro et al., 2019b) concerning the different georeferencing strategies. Although affected by a greater computational effort with a reduction of manual operations, the co-alignment procedure returns geometrically robust products (Table 3). Planar RMSE were in a range from +2.47% for the CO-MAR case in DG to -7.44% for the CO-OCT case in IG (20 GCP). The RMSE3D instead obtained all better values than the non-co-aligned cases, in particular the best result was always obtained in the CO-OCT case with a value of -6.84%. Systematic errors are spatially consistent over time, aided by the search for matched tie points on multiple timestamps, and thus may be discriminate for change detection tasks.  Table 2. Summary of points included in the Sparse Point Clouds (SPC) and Dense Point clouds (DPC) for the three different setups (0, 10 and 20 GCPs) per each analysed subchunk. Percentage rate (ΔN) between the amount of points contained in the co-aligned clouds and the original ones. The labels DEC, MAR, OCT indicate respectively the datasets of December, March, October; the labels preceded by the prefix CO the ones obtained by Co-Split.

ID
Conversely, by comparing the percentage changes between RMSEP and RMSE3D, it is evident that planar components affect more the final error than the vertical one since RMSE3D is slightly higher in all examined scenarios. Nevertheless, georeferencing strategies influence their results since their values improves with the increment of the number of GCPs, as already discussed by Saponaro et al., 2019b). Appreciable improvements, however, can be observed in DG cases by adopting the co-alignment methodology. In future work it will therefore be encouraging to evaluate possible improvements in DG cases by equipping RPAS technologies with RTK-GNSS receivers.  Table 3. Summary of statistics (mean error (ME), root-meansquare planar error (RMSEP) and three-dimensional error (RMSE3D)) measured on the 10 CPs distributed in the investigated scene. For each variable, the comparison between the models obtained from conventional and co-aligned procedures is presented in terms of percentage (Δ).

DoD outputs
DoDs is a 2.5D raster representation of the elevation grid variation between DEMs belonging to the same timestamp ( Figure 3). Uniformity of color corresponds to the zones with absence or extremely low value of uncertainty (δz is close to zero); conversely, the inhomogeneity in color is related to uncertainty δz on vertical component. The scalar fields in Figure 3 showed how the differences between the DEMs, returned by running the two alignment methods, attested to uncertainty values mostly close to zero. In line with the results reported by (Coulter et al., 2019), the most extreme values were traced in the outermost areas where noise and distortions are commonly non-attenuating, and, consequently, change detection studies should be focused on innermost areas. In the DG cases, the values increased and were not regular (see DEC case) due to the low inherited precision by co-registration. In fact, as noted in Table 3, the high variance associated with the low accuracy of the image geo-tags transferred a high degree of lability to the DG clouds. This caused an equally high δz value and therefore, although ME and RMSE values were improved (Table 3), these high uncertainties could affect the reliability of any change detection. Slight improvements were recorded in georeferenced cases with 20 GCPs compared to cases with 10 GCPs, demonstrating that except for highly accurate purposes it is possible to opt for this latter choice by reducing manual operations and processing time. The co-alignment methodology did not generate significant uncertainties along the vertical component, leaving coregistration between clouds essentially tied to the georeferencing strategy adopted.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition) Figure 3. Representation of the scalar fields 2.5D DEM of Difference (DoD) obtained by superposition of DEM processed by traditional approach or using co-alignment.

M3C2 outputs
M3C2 plug-in results were presented in Figure 4, providing the maximum distance, comprised between +0.752 and -0.752, among the clouds: points with no deviation are in green. Once again, the highest deviation values were detected on DG scenarios because of the tolerances of the receiver on board the RPAS. Nevertheless, being the distance uniformly distributed across the study area, it was recognized as a systematic error mainly due to the relative positioning between clouds which did not affect the morphology of the area. In all cases, the highest deviations have to be attributed to the noise caused by water reflectance and lack of coherence in the outermost areas. The case of the MAR dataset, georeferenced with 20 GCPs, was emblematic: larger deviations were recorded throughout the whole area under investigation, except for the coastal strip, which is green. As can be expected, the deviations tend to cancel out in the surroundings of the GCPs, demonstrating once again the importance of the distribution of GCPs in the surveyed area. This last test has therefore validated the results already obtained in the previous sections, confirming how the co-alignment methodology does not produce artifacts or distortions even on planar components. The morphology investigated is therefore not falsified, thus proving useful in the detection of changes.

DEC MAR OCT DG 10
GCPs 20 GCPs Figure 4. M3C2 maximum distance obtained by comparing dense point clouds derived from the two processing approaches for all georeferencing strategies.

CONCLUSION
The aim of this research work was to analyze the results of the sparse point cloud co-alignment process, an alternative method of co-registration among point clouds obtained from RPAS. For the detection of geomorphological changes based on multiepoch photogrammetric products from RPAS, accurate coregistration of point clouds is essential. Indeed, any misalignment of the models representing the analyzed scene could be detected as a false change or lead to unreliable descriptions. Image co-registration involves the identification of correspondence points between images, often GCPs distributed across the investigated scene, which simultaneously georeference and overlay the datasets to the same reference. The detection of these correspondence points has traditionally been done manually, requiring considerable human intervention, in terms of costs and time. Conversely, under this study, an alternative novel methodology was experimented.
Although conditioned by a greater computational effort but with reduced manual operations, the adoption of a co-alignment procedure between different datasets returns numerically consistent in point density terms and geometrically robust clouds. Systematic errors become spatially coherent over time because the clouds are the result of the same matched tie points on multiple timestamps, and thus be negligible for change detection cloud-to-cloud. Analysing the georeferencing strategy impact, it is possible to deduce that the largest contribution to the error improvement is due to the planar components. Overall, it was confirmed a relatively better behavior for each georeferencing strategy adopting the co-alignment methodology.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition) In differences raster between DEMs, it is evident how the area in all cases is characterized by uniformity of δz values close to zero, except for the outermost areas characterized by more extreme values and therefore higher values of δz. The high variance associated with the low accuracy of the image geo-tags transfers a high degree of lability to the DG clouds. This causes an equally high δz value and therefore although ME and RMSE values are improved, these high uncertainties can affect the reliability of any change detection. Even in the cloud-to-cloud comparison with M3C2 algorithms it is found the homogeneity of these deviations therefore showing that morphology is preserved in the co-alignment methodology but an improvement in cloud co-registration can only be obtained with a number greater than 10 GCPs implemented in the georeferencing phase. As can be expected, the deviations tend to cancel out in the surroundings of the GCPs, demonstrating once again the importance of the distribution of GCPs in the surveyed area. Given the accuracy and reliability of the results, which are comparable to the outcomes from classical approaches, the detection of geomorphological changes achieved by coalignment approaches may become commonplace in RPAS-SfM-based topographic change detection. The results obtained in this work show that a particularly large advantage of coalignment is that it forces poor quality surveys to achieve a more robust shared geometry, perhaps set by the other surveys, which strongly increases the comparability of surveys and the accuracy of topographic change detection. Future research work can therefore be set up to search for variations in geomorphological changes between multi-epoch datasets by adopting the two approaches analyzed in this paper.