EVALUATING THE QUALITY OF PHOTOGRAMMETRIC POINT-CLOUDS IN CHALLENGING GEO-ENVIRONMENTS-A CASE STUDY IN AN ALPINE VALLEY

Precise and accurate three-dimensional geospatial data has become increasingly available thanks to advances in both Terrestrial Laser Scanning (TLS) and Structure-from-Motion Photogrammetry (SfM). These tools provide valuable information for mapping geomorphological features and detect surface changes in mountainous environments. The exploitation of 3D point-clouds has been proven tremendously useful in the field of geosciences. It remains, however, controversial whether cost efficient photogrammetry can provide as accurate and reliable geospatial information as the significantly more expensive laser scanning or not. In this study, a rockfall case site in the territory of Obergurgl, Austria, is investigated in order to provide answers to the above question in a complex environment. The analysis includes different terrestrial photogrammetry configurations aiming to comprehensively define the strengths and limitations of terrestrial photogrammetry over TLS. The latter constitutes an optimized methodology that provides guidelines for costly future assessments as part of the site investigation phase in geohazard management. There are no doubts that compared to traditional and conventional surveying methods TLS and Photogrammetry both offer products much faster and with a much higher data density. In the current study, we show that when photogrammetry is applied following a well-defined optimized strategy, it can be potentially an adequate alternative to more costly TLS datasets for mass movement assessment and monitoring purposes. * Corresponding author


INTRODUCTION
Landslides and rock failures in mountainous environments have serious and dramatic impacts on society and infrastructure, often leading to fatal situations, especially in the Alpine terrain, where high touristic activity takes place throughout the year. Lately, the advancements of geoinformation enabled many researchers to incorporate remote sensing tools for precise visualization, assessment and monitoring of natural processes such as landslide and rockfall phenomena. As a result of the easy transportability and low operational costs compared to laser scanning surveys, close range photogrammetry has shown great adaptability in mountainous environment for collecting detailed 3D datasets. Rockfalls are generally initiated by multivariate climate and morphological factors that change the acting stresses on rock. Those events may include an increase in water table due to rainfall, erosion of layers, freeze-thaw processes in cold climates, chemical degradation or weathering of the rock or root growth (Corominas et al. 2014). In the past decade, terrestrial laser scanner (TLS) point-clouds have become a common practice to describe, monitor or characterize earth surface processes (Sack, Orme, 2013;Telling et al. 2017). However, their cost, maintenance, and operation may prove cumbersome and require wide resources, especially as surveys must be repeated for the study of dynamic processes (e.g., landslides, rockfalls, rock glaciers, etc.). As an alternative, dense 3-D point-clouds can be generated by applying structurefrom-motion (SfM-MVS) photogrammetry to photos taken by consumer grade cameras at a much lower cost (Scaioni et al., 2018, Eltner et al., 2016James, Robson, 2012). Both ground and Unmanned Aerial Vehicle (UAV)-based SfM photogrammetry have shown to represent a multi-scale, highresolution mapping tool for volcanology, geomorphology, structural geology, etc. (e.g., Bemis et al., 2014;Smith et al., 2016). In order to monitor the dynamic geomorphological processes, point-clouds from different epochs must be registered with high accuracy. When dealing with photogrammetry-based point-clouds, the registration phase becomes involved, especially in such dynamic environments. Therefore, the quality of the multi-epoch point-clouds may deteriorate and introduce biases and artefacts into the monitoring process. Assessing these failures and enable the cheaper usage of such technique (SfM), is a keystone for improving the suitability of photogrammetric techniques for both research and professionals. In this work, we propose a quality control scheme, where the new photogrammetry-based point-clouds are assessed as to their suitability for monitoring purposes. Literature shows that the quality and accuracy of point-clouds is evaluated both subjectively and objectively. Subjective evaluation is carried by users that rate the level of the point-cloud impairment, with respect to reference data. Objective quality measure commonly includes accuracy metrics such as distances of point-to-point and point-to-plane with respect to reference data. However, most research was carried either in urban or closed environments, where there is little regard as to the relative quality between point-clouds. In this paper, we investigated an active mass movement and propose a quality assessment procedure for multi-epoch point-clouds in complex environment while testing acquisition and processing parameters of the photogrammetric point-clouds for geomorphological monitoring. Images were taken at different configurations and were processed in both open-source and proprietary software. We assess the quality of the resulting point-clouds to a laser scanned point-cloud and to each other. Identifying morphological changes in hazardous areas is an important task for local and regional protection authorities.

QUALITY METRICS
Quality assessment of 3D point-clouds has been focusing mostly on points-clouds that were acquired under a controlled environment, where well defined objects can be positioned or used Ebrahimi and Alexiou, 2017;Zhang et al., 2015). Quality is usually assessed by subjective and objective metrics. Subjective evaluation is commonly based on visual inspection, usually carried by a testgroup that assesses the point-cloud completeness and its density Ebrahimi and Alexiou, 2017;Zhang et al., 2015). More objective approaches usually compare the point-clouds in reference to control points or to well-defined objects (Moon et al., 2019;Zhang et al., 2018). Both approaches are therefore comparative use other point-clouds for reference. Notably, the documented methods are confined to either laboratory conditions or urban environment, where well-defined objects are used for the assessment, which can be positioned anywhere in the scene. However, natural features often represent much more complex setting: using test groups to evaluate each dataset is immaterial, while fixing control points or other objects is not always possible. Aiming for an independent quality assessment procedure that can be used as a preliminary step for geomorphological analysis, we propose an assessment workflow that is not restricted to specific settings, and still maintains the comparative nature of both suggested approaches. All point-clouds are first registered into the same reference system, defined by the most reliable dataset. Then, all of the analyses are carried out both for the entire point-cloud and on selected regions in the cloud, which were chosen arbitrarily and are scattered throughout the scan. Note that those regions should appear in all analysed point-clouds, to facilitate comparison.

Objective evaluation
The quantitative measures are divided into two groups: those that are internal to the point-cloud and can be measured for each cloud by itself, and those that are external and require a reference point-cloud. Notably, the internal measures require a reference point-cloud for comparison. These include the following: ▪ Number of points, i.e. the total number of points in the point-cloud over a specific area.
▪ Number of neighbours, estimated by the average number of points within a sphere with radius R.
▪ Surface density, measured by the number of neighbours divided by the neighbourhood surface: With N the number of neighbours in a sphere of radius R. The density measure is also colour-coded and visualised to analyse its uniformity throughout the cloud/region. ▪ Surface roughness is estimated pointwise for local regions, as the distance between each point to the best fitting plane computed on its nearest neighbours. The total roughness value is the mean value of all points' roughness in that section. Here we measure the roughness for two sphere radii, to achieve a more robust measure.
▪ Plane orientation is defined if a plane divides 3D space into two half spaces. Because the surface normal is perpendicular to the plane, it always points to one half space. Thus, the normal vector distinguishes the two areas, by always pointing to exactly one of them.
The external measures of each point-cloud are evaluated based on a reference point-cloud, which is considered the most reliable of the available datasets: ▪ Point-to-point distance is defined as the average distance between the analysed point to the reference. Although this measure mostly reflects the registration error, it also mirrors the relative density of the analysed cloud to the reference, throughout the cloud. ▪ Distance between profiles is defined as the average distances between corresponding points along with the profiles in the analysed and reference point-clouds. The distance is measured orthogonally to the local trend.

Subjective evaluation
The subjective evaluation is carried out visually, by comparing the completeness of the point-cloud at certain regions. Following , the evaluation is carried out by interaction with the content by composite movements of mouse (left, right, up, down, zoom) while providing a score, with no time restriction. As the presence of additional attributes (such as colours) can be considered distracting to the evaluation of geometrical errors , no colour values were assigned to the points. In addition, a visual inspection of the point cloud profiles was carried out, aiming to estimate the similarity of density and smoothness.

ASSESSMENT OF PHOTOGRAMMETRIC POINT-CLOUDS
In the current study, we applied the proposed workflow to assess the quality of photogrammetry-based point-clouds. The aim was to estimate the quality of different configurations, both in processing and acquisition, for geomorphological usage in a complex setting.

Study site & data acquisition
A rockfall close to the village of Obergurgl, Austria (46°51'59"N, 11°00'60"E, Figure 1) has been chosen to evaluate the quality of photogrammetric based point-clouds. This rockfall is defined by an area of approximate ~110 m height and ~60m width. It is formed in Triassic and Cambrian metamorphic rocks (meta-schists and gneiss formations). Images were acquired on two succeeding days sharing similar lighting conditions with a DSLR camera (Canon EOS 6D) equipped with a zoom lens and a full-frame CMOS sensor (36 x 24mm) with 6240 x 4160 pixels ( Figure 2). Due to the steep terrain and the difficult accessibility of the site, the images were taken ~100 m away from the modelled surface on a path facing the rockslide, over its middle section. Fifty-seven images were acquired with a 24mm focal-length lens, and 47 with a 105mm configuration (i.e. maximal zoom length), the latter acquisition focusing only on the upper part of the rockslide (Figure 3 and Figure 4). The reference point-cloud is a terrestrial laser scan, acquired with a Riegl VZ-2000 on the first day. The average point spacing was set to 3 cm. Measurements were taken from three different scanning positions, roughly within the same distance to the rockfall site as the ground photos. The location of each scanning position was georeferenced using differential GNSS. It should be mentioned that no manmade control points were placed on the rockfall site due to its inaccessibility. The resulting uncertainties will not play a vital part in the model since the morphological signs of mass movement are in larger magnitude from the Ground Sampling Distance (GSD).

Photogrammetric processing
To acquire comparable datasets, we produced multiple pointclouds from the same collected images by means of two photogrammetric software products: Agisoft Metashape Professional v1.5, , a proprietary stand-alone software (Agisoft, 2019) and a combination OpenMVG (Moulon et al, 2016) and OpenMVS (Cernea, 2015) open-source libraries. Agisoft Metashape is one of the most commonly used photogrammetric software for geomorphological surveys (Eltner et al., 2016;James et al., 2017). Detailed descriptions of photogrammetric workflows for the production of point-clouds using Agisoft Metashape Pro can be found e.g., in Turner et al. (2015) and James et al. (2017). OpenMVG provides methods to solve camera orientation and calibration problems, by implementing keypoints matching across images and subsequent bundle adjustment. OpenMVS is used afterwards to produce depth maps and dense point-cloud via a patch matching approach (Bleyer et al, 2011).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) Different acquisition configurations were tested by varying the: number of images (8, 30 and 47) and the focal lengths (24mm and 105mm). Each acquisition scenario was processed independently with both commercial and open-source software to indicate potential limitation and strengths of each configuration and to test our quality control method (Table 1). In total, six datasets were created. Following their creation ( Figure 5), all point-clouds were registered to the TLS data using the semi-automatic registration tool in Cloud Compare (CloudCompare, 2019). Note that dataset no. 6 represents only of a small region within the entire rockfall and it was analysed accordingly.
Figure 5. Schematic representation shows the flow of the proposed methodology.

Results
To assess the quality between different photogrammetry-based point-clouds, and as well as assess the suitability of such pointclouds for geomorphological usage. Generally, we applied the proposed workflow on the photogrammetric datasets, while using the TLS data as reference. Table 2 describes quality measures for one sub region of the rockfall (upper section). The radius in which the analyses were made is 5cm. Note that also the reference dataset has been analysed, to provide comparable information. It can be seen that the point density of dataset no. 6 is ~2.5-fold than that derived from the TLS, and almost 11 times larger than the one acquired by 30 images with focal of 24mm and processed with the opensource software. Notably, the proprietary software did not produce much better results, with 6.200 point per square-metre, as opposed to 5.740. This is also reflected by the number of neighbours in a 5cm sphere. Initially, individual point cloud metrics have been assessed such as the total number of points (Table 1), and the pointcloud density. Then, pointclouds were assessed among each other by comparing with the TLS reference. Thirdly, a geometrical approach assessment has been carried out by implementing a cross section among the models to evaluate their respective points scattering. The latter theoretically should remain stable between all datasets. In Figure 6 is shown a comparison of the point-clouds with their respective neighbour density in 5cm radius. A small region common for all the models, has been selected and cropped for further analysis using above describe quality metrics. Different metrics are presented in Figure 7, Figure 8 and Figure 9 for the respective 3D pointclouds.    Moreover, a geomorphological cross section ( Figure 10) has been designed and overlaid on the multiple resulted 3D models for detailed inspection. In the current section, we computed a cross section as the intersection between the produced mesh model and a specific plane surface. The latter comprises a characteristic rockfall scene as it represents the active source zone. It can be seen that in individual parts of the scene the 3D models show a multivariate scattering dependent on the selected configuration compared with the reference TLS. For instance, regarding the software used for SfM, Agisoft Metashape showed an increased point scattering from the reference when compared with the open source one.

CONCLUSION
The current attempted to illustrate strengths and weakness of low-cost terrestrial photogrammetric surveys against laser scanning. Results show that our procedure provides a reliable tool for detailed point-cloud assessment, and thus facilitates the usage of photogrammetric based 3D data to monitor rockfall phenomena at low costs. The study has been mainly focused on geomorphological failures on complex environmental setting with limited accessibility to the site. Results indicate that following a concrete structure of image collection and SfM techniques, terrestrial photogrammetry could be applied for mass movement assessment in spatial resolutions close to Laser scanning. It should be clarified that limitation in regards of the passive sensor's cons are still existing such as occlusions due to lightning conditions and should be handled with care in the analysis phase. As a result, the proposed photogrammetric concept proved effective in terms of quality and operational costs and it could be an alternative for site inspection of mass failures. Using both photogrammetry and lidar can lead to higher efficiency for multivariate projects. LiDAR should be certainly used for mapping sites below tree canopy. From the other side, photogrammetry will be the optimal choice for projects that require detailed visual data. Both technologies can bring crucial data and the choice of method depends mainly on the particular case as well as time, budget, and capturing conditions. Further research should focus on the ability of detecting surface changes among different epochs, as well as how reliable is a method in helping the analyse of failure mechanism.