INVESTIGATING THE PERFORMANCE OF A HANDHELD MOBILE MAPPING SYSTEM IN DIFFERENT OUTDOOR SCENARIOS

In recent years, portable Mobile Mapping Systems (MMSs) are emerging as valuable survey instruments for fast and efficient mapping of both internal and external environments. The aim of this work is to assess the performance of a commercial handheld MMS, Gexcel HERON Lite, in two different outdoor applications. The first is the mapping of a large building, which represents a standard use-case scenario of this technology. Through the second case study, that consists in the survey of a torrent reach, we investigate instead the applicability of the handheld MMS for natural environment monitoring, a field in which portable systems are not yet widely employed. Quantitative and qualitative assessment is presented, comparing the point clouds obtained from the HERON Lite system against reference models provided by traditional techniques (i.e., Terrestrial Laser Scanning and Photogrammetry).


INTRODUCTION
Nowadays, the increasing availability of accurate and efficient surveying technologies allows cost-effective acquisition of high-resolution 3D data of both internal and external environments. The well-established Terrestrial Laser Scanning (TLS) and Photogrammetry techniques have been recently joined by portable Mobile Mapping Systems (MMSs), which are especially revolutionizing indoor mapping. Thanks to their low weight, handheld, backpack and trolley instruments can be easily carried by a person, who performs the survey by simply walking through the area of interest (Nocerino et al., 2017, Otero et al., 2020. These devices are particularly suited to support facility management activities that require fast data acquisition to capture the state of a building at a certain time. The obtained point clouds serve, e.g., as starting point to create the Building Information Model (BIM) of the structure, fundamental for building maintenance and renovation (Cantoni andVassena, 2019, Comai et al., 2020).
One of the main advantages of portable MMSs is that they are able to leverage on Simultaneous Localization and Mapping (SLAM) algorithms to make up for Global Navigation Satellite System (GNSS) unavailability (Di Filippo et al., 2018). Developed at first by the robotics community to allow the motion of a robot in an unknown site through the measurement of a limited number of landmarks, SLAM methods have been subsequently customized to retrieve the trajectory of a moving sensor (such as a laser scanner) and to simultaneously build an accurate map of the surveyed environments (Moosmann and Stiller, 2011). Two main approaches can be applied to solve the SLAM problem (Grisetti et al., 2010). The first one, also known as on-line SLAM, includes filtering methods that incrementally add new measurements as they are available and estimate in real-time both the device position and the map. The full SLAM approach, instead, computes the trajectory from the entire set * Corresponding author of measurements acquired during the survey, detecting and exploiting also loop closures. Following a trajectory that contains close-loop paths (i.e., re-surveying some areas), allows to add consistency constraints in the global SLAM optimization problem (Lu and Milios, 1997), minimizing error accumulation and producing more accurate 3D point clouds of the surveyed environment.
The aim of this work is to assess the performance of a commercial handheld MMS in different outdoor scenarios, comparing the acquired point clouds with the ones provided by standard techniques, i.e., TLS and Photogrammetry. In particular we present two case studies: the survey of the outside of a large building and a torrent reach. The latter is one of the first attempts to investigate also the applicability of portable systems for natural environment monitoring.

RELATED WORKS
In the literature, several in-depth analyses have been presented that highlight advantages and limitations of portable systems when mapping buildings, civil structures, cultural heritage and urban areas.
A thorough experimental evaluation of portable devices is described in (Nocerino et al., 2017), where handheld and backpack systems are used for the indoor inspection of a building and for the outdoor survey of a city square. The tests in the indoor scenario point out the higher noise that characterizes handheld and backpack instruments with respect to TLS. For the outdoor case, instead, the comparisons between the portable MMS point clouds and the reference one, acquired through a MMS mounted on a vehicle, show tight correspondence, with mean signed differences in the range 1-2 cm. The behavior analysis of two wearable indoor mapping systems carried out in (Lagüela et al., 2018) reports encouraging results, with mean absolute distances of 10 cm between the portable MMS data and the reference TLS point cloud. These outcomes are in agreement with the ones reported in (Tucci et al., 2018). However, the latter work also highlights weaknesses of SLAM-based systems. Architectural details, which are clearly recognizable in the static cloud, are less visible in the portable MMS data, characterized by lower density and higher noise. Moreover, artifacts (i.e., double surfaces) were produced by some of the tested devices, with residual misalignments between the data acquired on the same area at different time.
Several studies have been devoted to validate portable MMSs also in underground environments (Masiero et al., 2018, Sammartano andSpanò, 2018), including tunnels and mines (Raval et al., 2019), where the absence of GNSS signal, the context complexity and the difficult illumination conditions make it challenging the use of other surveying techniques. Experiments described in these works bring out the importance of the trajectory followed during data acquisition: the accuracy and the precision of the final point clouds, indeed, are significantly influenced by the presence of closing loops that avoid drift error propagation.
In the last few years, portable MMSs are emerging as valuable technology also for forest inventory application (Balenović et al., 2021). Studies on this topic report centimeter accuracy in the estimation of individual tree attributes (e.g., stem location, height and diameter) from point clouds acquired with handheld laser scanners (Chen et al., 2019, Gollob et al., 2020. The logistical advantages and ease of use of lightweight portable devices suggest their applicability also for natural environment monitoring and complex topography survey (James and Quinton, 2014). However, less attention has been given to these systems in the field of geomorphology. To the best of our knowledge, (Williams et al., 2020) is the only work that carries out an in-depth quantitative evaluation of the performance of a backpack system in a river environment. The described results are promising, with a mean vertical error of the Digital Elevation Model derived from the portable laser scanner survey of less than 2 cm. In this paper, we will further investigate the use of a handheld MMS in a fluvial environment, at the reach scale.

MATERIAL AND METHODS
The portable MMS tested in this study is the HERON Lite handheld device by Gexcel srl (Gexcel srl, 2021). It is composed of a Velodyne Puck LITE laser scanner, with 16 channels that emit infrared laser beams at a wavelength of 903 nm and allow to acquire 300,000 points/second in single return mode from a 360 • horizontal field of view and a 30 • vertical field of view. Range measurements are performed through the time-of-flight principle, with maximum measurable distance of 100 m. The laser scanner sensor is coupled with a XSens MTI Inertial Measurement Unit (IMU), whose data are exploited in the system trajectory estimation. While performing the survey, the scanning head is mounted on a telescopic carbon fiber pole and connected to a battery and a control unit (Fig. 1).
According to the manufacturer's specifications, the system provides a local accuracy of 3 cm and a final global accuracy of 5 cm, that can downgrade to 20-50 cm depending on the effectiveness of the SLAM algorithm (Gexcel srl, 2021). This can be significantly affected by the presence of loop closures on the trajectory and by the characteristics of the scanned environment. Raw data are processed through HERON Desktop software, which computes the trajectory and the map in a three-step workflow. The first stage, known as Odometer, corresponds to an on-line SLAM approach (see Section 1) and performs a first estimate of the sensor trajectory. It retrieves the pose of each cloud (a single 360 • scan) by registering it to the previous ones via the Iterative Closest Point (ICP) algorithm and leveraging also on the IMU data. Since this method can accumulate drift on long trajectories, subsequent refinement is required. To this end, the Create maps step is applied to split the full trajectory into local maps, that represent the input to the final Global Optimization stage. The algorithm implemented in this module follows a full SLAM approach and performs a simultaneous global registration among all the local maps, allowing to exploit loop closures and minimizing misalignment errors among overlapping clouds. Finally, the obtained complete map can be automatically filtered to remove moving objects (e.g., walking people or moving vehicles) applying the Clean Data process.
As previously mentioned, in this work the performance of the HERON Lite system was investigated on two different outdoor scenarios, described in the following. The 3D point cloud retrieved from the handheld MMS survey was compared to a reference (ground-truth) model, acquired by a TLS for the first case study and produced using the photogrammetric technique in the second one. In both cases, the HERON Lite model and the reference one were preliminary aligned via the ICP algorithm.

University building
As first case study, we performed the outdoor mapping of the main building of the Rizzi University Campus in Udine (Italy). The structure has a rectangular base measuring approximately 200 m × 110 m and it is composed of three floors.
The HERON Lite survey lasted 26 minutes, for a total trajectory length of 1470 m. Walking speed was reduced in transition areas (i.e., moving from one side of the building to the other) and three closed loops were performed while scanning the environment. Significant drift (> 1 m) was visible on the point cloud produced by the on-line SLAM algorithm, that was subsequently reduced in the Global Optimization step by taking into account the closing loop constraints. The complete point cloud was further post-processed inside the JRC 3D Reconstructor software, in order to automatically remove isolated points. Figure 2 shows the X-ray orthophoto of the surveyed building, extracted from the final model by projecting the point cloud on the horizontal plane. To evaluate the accuracy reached by the handheld MMS, a model of the south facade was acquired by means of a Faro CAM2 Focus TLS. This instrument covers a 360 • × 300 • field of view and allows to measure up to 976,000 points/second with high accuracy (3.5 mm @25 m). The facade, which has a total length of 203 m, was surveyed through five static scans carried out from different stations and subsequently registered exploiting a set of artificial spherical targets.

Vegliato torrent
The second case study is represented by a reach of the Vegliato torrent, placed in Gemona del Friuli (Udine, Italy). The study area covers approximately 4000 m 2 and is characterized by the presence of two check dams, steep river banks and fluvial terraces. These vertical elements are challenging surfaces for the survey.
Due to the irregular morphology of the riverbed, data collection with the handheld MMS was slower than the previous case: it took 30 minutes to complete the survey, for a total trajectory length of 680 m. Also in this case, three closed loops were performed, that were exploited in the last stage of the data processing to reduce misalignment errors among local maps. As done for the University building model, isolated points were automatically removed from the final point cloud using the functions of JRC 3D Reconstructor software. Furthermore, points with a normalized intensity value less than 0.10 were filtered out, in order to delete the majority of vegetation points. Figure 3 shows the X-ray orthophoto produced for the surveyed area.
To obtain a reference model, two flights were performed with a professional Unmanned Aerial Vehicle (UAV) (DJI Matrice210v2 quadcopter), acquiring photos with a DJI Zenmuse X5S camera (5280 × 3956 pixels, 25 mm focal length). The first flight was carried out in programmed mode at an The entire image set at full resolution was then processed by the photogrammetric software 3DF Zephyr to produce a dense point cloud of the study area. The GCPs were introduced in the bundle adjustment as constraints, whereas the residuals on the CPs (i.e., the differences between GNSS-measured coordinates and photogrammetric ones) were used to evaluate the model accuracy, obtaining a Root Mean Square (RMS) error of 3.2 cm.

University building
At first, we focus on the outcomes achieved for the University building case. As initial quantitative comparison between the portable MMS model and the TLS one, the point-to-point absolute distances between the two point clouds were computed with CloudCompare 1 . Please note that, to avoid unrealistic values for points belonging to surfaces not scanned by one of the sensors, only distances less than 35 cm were taken into account. Figure 4 shows the results for the left side of the facade (the right one has a symmetrical behavior). The average cloud-tocloud (C2C) distance, estimated for the whole facade, is 7.3 cm, with larger deviations (mainly in the altimetric direction) at the ends, where values of 25 cm are reached. These discrepancies are further highlighted by the vertical sections shown in Fig. 5, obtained extracting 10 cm-wide slices from the clouds at distances d = {2, 30, 60, 100} m from the leftmost point of the facade. Section at d = 2 m reveals vertical distances of magnitude ∆Z = Z TLS − Z HERON = +25 cm, that decrease to ∆Z = +4 cm at d = 30 m. The relative deformation between the two models leads to ∆Z = −11 cm at the center of the facades.
A possible interpretation for these errors lies in the drift that was not completely removed during the full SLAM processing step (Global Optimization module, Section 3), probably due to  From a visual inspection of the analyzed point clouds, it was easy to notice that the handheld MMS and the TLS sensors are characterized by different noise levels. As suggested in (Nocerino et al., 2017), to quantitatively assess this feature three portions of the point clouds were sampled: on the wall of the ground floor, on the wall of the first floor and on one side of a column, respectively. A plane was then fitted to each subset, computing the RMS of the distances of the points from the estimated plane. The results reported in Tab. 1 show that the TLS significantly outperforms the handheld system, as expected. We must underline that these values are in accordance with the technical specifications of the instrument: the RMS of the residuals for the HERON Lite points ranges from 1.6 cm to 2.4 cm, slightly lower than the nominal precision (3 cm). To investigate the influence of the HERON Lite accuracy and precision on the final cartographic products that can be derived, we manually vectorized the horizontal section of each column from both the MMS and TLS point clouds projected on the horizontal plane. Then, we measured the planar distance D between each pair of digitized adjacent columns (Fig. 6) and evaluated the differences between measurements performed on the MMSderived and TLS-derived products, obtaining a RMS of 3.8 cm. This is a further confirmation that the handheld MMS technology is suited to efficiently derive the blueprint of a building with an accuracy compatible with several applications. completeness. As can be seen from Fig. 7(b), point clouds obtained from a static laser scanner survey can often present data gaps at occluded surfaces, i.e, surfaces hidden by objects in the line of sight of the laser beam. To avoid these issues, several scans from different points of view should be acquired, increasing time and cost of both data collection and processing. On the contrary, the continuous sensor movement (and the consequent variation of the point of view) that characterizes portable MMS data acquisition allows to easily survey all the surfaces (if a suitable trajectory is followed).

Vegliato torrent
Similarly to the previous case study, the first quantitative comparison between the HERON Lite point cloud of the Vegliato torrent and the photogrammetric (reference) one was performed by computing the absolute point-to-point distances. Results are shown in Fig. 8: the average cloud-to-cloud distance is 5.4 cm (approximately double the error of the photogrammetric model estimated on the CPs), with larger deformation up to 20 cm only in a small part of the area (lower right, in yellow). Spots characterized by higher errors (in red), instead, are mainly low vegetation points (e.g., small bushes and isolated shrubs) that were not filtered from the point clouds.
In contrast to the University building case, for which the differences were mainly distributed along the altimetric component, here geometric deformations are visible above all on the XY plane. Figure 9 gives further insight on the discrepancies between the two models, showing three 10 cm-wide vertical sections extracted at the spillway of the upstream check dam, at the left wing and in the middle of the riverbed. For all the sections, altimetric differences are barely visible, whereas planimetric distances (∆XY ) ranges from 4 cm to 14 cm.
Although the density of the handheld MMS cloud is higher than that of the photogrammetric one (1605 points/m 2 and 1058 points/m 2 , respectively), the sections reported in Fig. 9 and the detail of the point clouds shown in Fig. 10 clearly demonstrate the higher noise that affects the HERON Lite model, as already seen in the previous case study. In Fig. 10(a) only the larger boulders can be distinguished, whereas in the photogrammetric point cloud ( Fig. 10(b)) smaller details can be appreciated. Therefore, reliable values of surface roughness could be difficult to obtain with the handheld MMS.

CONCLUSION
In this paper we investigated the performance of the HERON Lite handheld MMS in two different scenarios: the outdoor mapping of a large building and the survey of a torrent reach. The first case study confirms the outcomes described in previous works, that reported accuracy ranging from 5 to 10 cm for the survey of civil structures or urban areas with portable systems. At present, this technology does not appear to be suitable for applications that require high precision measurements; however, it represents a valuable and cost-effective solution for fast surveying large sites.
In the field of natural environment monitoring, portable MMS have not yet found widespread application and a limited number of studies have evaluated the technology in this context. For the case study of the Vegliato torrent discussed in this paper, differences estimated between the photogrammetric model and the HERON Lite one are encouraging: there is evidence that a portable MMS could be a valid alternative to Photogrammetry for natural environment mapping, especially in zones where flying with UAVs is denied or when the GNSS survey of GCPs (which proved to be fundamental when applying camera selfcalibration procedure) is not feasible. Future research should focus more deeply on evaluating portable systems for geomorphological applications, defining best practices for data acquisition and processing also for these scenarios.  Gollob, C., Ritter, T., Nothdurft, A., 2020. Forest inventory with long range and high-speed personal laser scanning (PLS) and simultaneous localization and mapping (SLAM) technology. Remote Sensing, 12(9), 1509.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2021 XXIV ISPRS Congress (2021 edition)