LIDAR IMMS VS HANDHELD MULTICAMERA SYSTEM: A STRESS-TEST IN A MOUNTAIN TRAILPATH

Indoor Mobile Mapping Systems (IMMS) are attracting growing attention, especially when LiDAR sensors are considered, thanks to the possibility to obtain wide range and complete data in those areas where GNSS signal is not available. However, the drift error that accumulates during the acquisition is often inadequate in the absence of quality constraints in case of extensive acquisitions. Concurrently, recent developments regarding multicamera mobile solutions have shown promising results in containing the drift error, but data produced are too noisy and not enough complete in terms of acquisition range. This paper compares a Laser Scanner IMMS and a multicamera system in a stress test concerning the survey of a complex and extended route. The two systems are the Laser Scanner Backpack IMMS Heron MS Twin Color produced by Gexcel and a laboratory prototype of a handheld photogrammetric multicamera named Ant3D. The objectives are to calculate and compare the drift errors and to evaluate the quality of the produced point clouds. Quantitative results demonstrate that the drift error per meter of trajectory for the Heron Backpack is 10 times greater than the one of the multicamera. From a qualitative aspect, Heron Backpack generates 3D data in a wider range, allowing a more complete reconstruction of the environment when compared to the multicamera system one. On the other hand, the encumbrance and manoeuvrability of Ant3D make it more versatile in surveying very narrow spaces.


INTRODUCTION
The field of geomatics is experiencing a trend of technological innovation favouring point cloud (PC) data acquisition on the move, mainly by LiDAR (Light Detection and Ranging) sensors. This trend has also accelerated due to improved sensors and positioning algorithms that gradually increase the output accuracy. In the last years, they have made possible the development of many commercial wearable devices that can be successfully used in indoor environments (Indoor Mobile Mapping Systems, IMMSs) (Otero et al., 2020), or, more generally, in those areas where GNSS signal is not available, like densely vegetated areas (La Placa & Doria, 2022), underground sites (D'Agostino et al., 2022) and urban canyons (Blaser et al., 2020). Especially in GNSS denied areas, Simultaneous Localisation And Mapping (SLAM) algorithms, at the base of LiDAR IMMSs, allow for the simultaneous estimation of the sensor trajectory and the creation of a map of the environment. This is achieved by combining an Inertial Measurement Unit (IMU) and scans acquired and automatically registered along the trajectory. It is recommended to set up a survey scheme made of closed loops to create some rigid constraints to adjust horizontal and vertical errors and refine the point clouds creation (Chiabrando et al., 2019). However, even when such a scheme is followed, the accuracy achievable with modern IMMSs is not easily predictable and is often inadequate in the absence of quality constraints in extended acquisitions. This is the case of those surveys where it is not possible to reinforce the trajectory by closing the loops, like paths starting and ending in different points. In essence, even when the acquired data in terms of * Corresponding author density, completeness and noise of the PC meet the requirements, the drift error that accumulates during the acquisition limits the applicability of these systems. In a previous test carried out by the authors employing the Backpack IMMS Gexcel Heron MS Twin Color, it was shown that by using well-distributed ground control points along a straight path of about 500 m the drift error was contained within 50 cm, whereas it reached 13 m if not constrained (Marotta et al., 2022). Also the use of Zeb-REVO to map an underground coal environment in a linear trajectory with single loop revealed a drift error which, although not quantified, makes the result unsuitable (Raval et al., 2019). In this framework, recent developments regarding fisheye closerange photogrammetry and multicamera mobile solutions have shown promising results in containing the drift error in extended acquisitions. Troisi et al. (2017) showed the effective use of fisheye photogrammetry using frames extracted from video sequences to survey an extensive underground tunnel environment with the goal of locating a point at the very end, no constraints were used along the path, they report a positioning error of around 7 m for a 1 km long trajectory. While Perfetti and Fassi (2022) presented a test carried out with a handheld fisheye multicamera system to survey extensive confined spaces where they evaluated the drift error resulted from several open-loop acquisitions, the results reported a drift of around 4-5 cm for every 100 m of acquisition. This shows that the accuracy achievable employing image-based solution exceeds the accuracy reported in the literature from IMMS, However, the characteristics of the produced PC often do not meet the requirements in terms of noise and completeness of the data over long ranges, since the quality of the PC produced by dense image-matching is dependent on: (i) image quality, i.e. lighting, exposure conditions and sharpness; (ii) object features, i.e. texture quality and (iii) Ground Sampling Distance (GSD): all conditions that are difficult to control and keep constant in complex 3D scenes such as the outdoor environment imaged with fisheye lenses. For instance, the GSD fast degradation with increasing distance, especially using fisheye lenses (Perfetti et al., 2017) limits the PC range significantly.

Paper's goal
The state of the art involving IMMSs is characterized by the presence of contributions comparing the achievable accuracy and quality of point clouds obtained by such technology. To give an example, among the main contributions here we reported the work of Tucci et al., (2018), where three IMMSs are analysed from both a qualitative and quantitative aspect, and also the one from Lehtola et al., (2017) in which five commercial IMMSs and three research prototypes are compared and evaluated. As already stated, literature is mainly dedicated to IMMSs equipped with Laser Scanner (LS) sensors. Contributions testing image-based instruments are considerably fewer, given the limitations inherent to a passive technique with respect to an active one such as the LS and the range of acquisition. In the IMMSs review from Otero et al., (2020), only 4 of the 21 analyzed devices are imagebased. A recent study in this field is the one by Ortiz-Coder & Cabecera, (2021), in which a prototype that exploits video-based photogrammetry is described and tested. Based on the current literature, we can finally conclude that there is a lack of contributions comparing range-and image-based IMMSs. Therefore, the present paper wants to compare an IMMS and a multicamera system based on two different acquisition methodologies in a stress test concerning the survey of a complex and extended route. The two systems are the Backpack LS IMMS Heron MS Twin Color produced by Gexcel ( Figure 3) and a laboratory prototype of a handheld photogrammetric multicamera equipped with five RGB cameras and fisheye lenses (Figure 4, patent pending No. 102021000000812). The instruments are tested in the same case study to quantify the results in terms of positioning accuracy and quality of the produced point cloud in terms of density, range and noise. The comparison constitutes a challenging stress test because: there is no possibility of performing close loops to reinforce the trajectory when dealing with SLAM algorithm; the geometry characterizing the environment is limited to very few elements, that are outcropping rocks and trees trunks and, finally, it is not possible to acquire well distributed GNSS points to help the trajectory reconstruction given the presence of dense vegetation. Moreover, the presence of a very narrow and a vast cave gives us the possibility to deal with several complex scenarios to assess their mutual limits and advantages.

Case study
The case study chosen for the test is a mountain footpath located near the villages of Prosto and Piuro (SO), in the Chiavenna valley, in the northern part of Lombardy region. It is about 2.5 km long, with an altitude difference of approximately 350 m, starting from 700 masl and ending at 360 masl. The side along which it winds has an average slope of 34° and it is covered with very dense vegetation. Evident traces of geomorphological events with glacial origin are present in the whole area, like erratic boulders and very smoothed rocks forming slides. These peculiarities are due to the shaping action of the huge ice flow descending along the entire Chiavenna valley during the last glaciation. Natural caves called "Trone", used for soapstone extraction, are present along the path ( Figure 2). Two of these caves are particularly narrow, barely allowing the passage of a person. At the same time, the last one is considerably larger. At the beginning of the route, 4 scans were acquired with a Terrestrial LS as constraints for the two acquisitions. Moreover, some RTK GNSS points were measured along the trail to be used as checkpoints when the vegetative cover allowed for signal reception.

Gexcel Heron MS Twin Color
The first described instrument used for the test is the Backpack LS IMMS Heron MS Twin Color ( Figure 3) produced by Gexcel (Gexcel, 2022). For the sake of brevity, from now on the instrument is abbreviated with "Heron Backpack". The instrument and processing workflow are described in detail in Marotta et al., (2022), where its accuracy was improved when tested to survey an urban center. Therefore, here are reported only the main features. The IMMS does not use GNSS information to reconstruct its trajectory, relying on SLAM technology. It is equipped with an IMU and two tilted Velodyne Puck LITE LiDAR sensors for robust 3D geometry acquisitions. The declared local accuracy is about 3 cm. The instrument is also equipped with a full-resolution panoramic RGB camera to acquire images to color the point cloud. If necessary, especially in dark places, it is possible to mount on the pole a ring LED light, to obtain correctly exposed images. Technical specifications of the instrument are reported in Table 2. As described in Marotta et al., (2021), the survey took two different days, one in October 2020 and the other one in March 2021, because of storage issues faced during the first day. A total of 7 trajectories were acquired, stopping the acquisition when the operator needed to rest and taking about two hours to complete the survey. A summary of the trajectories' characteristics is reported in Table 1.  The Odometer phase required a continuous change of parameters to make the best use of the poor geometry of the forest environment to correctly reconstruct the trajectory. This was very time-consuming, as any discrepancies between the IMU and the alignment of the scans had to be observed in real time and eventually adjust. The Global Optimization step reduces the drift effect by creating links along the trajectory. It took several iterations since the ICP (Iterative Closest Point) algorithm was put to test by the presence of dense vegetation, which caused lots of homologous points to be found in the foliage. Also, the acquisition performed in two different periods of the year stressed the algorithm considerably, changing the environment configuration and degrading the achievable accuracy.
In this phase, it is also possible to insert static scans to better constrain the IMMS point cloud. At this purpose, four scans acquired with Terrestrial LS FARO Focus 3D X330 in correspondence with the final part of the path, i.e., the square of Prosto village, were added to the process, after being georeferenced by means of 12 RTK GNSS pointsmean registration error: 0.028 m. At the end of the iterative process, data were exported to Reconstructor, proprietary software developed by Gexcel, selecting a range of 30 m around the sensor and subsampled to 5 cm. The final point cloud counted about 2.5 millions of points.

Ant3D
The second tool used in the comparison is a prototype of a photogrammetric multicamera developed for rapid, on-the-move 3D surveying of confined and elongated spaces, such as tunnels, galleries, passages, corridors, caves, and forest trails called Ant3D (Perfetti and Fassi, 2022;patent pending N. 102021000000812). The device consists of a hand-held instrument housing five cameras with fisheye lenses, and a backpack housing a computer and the battery for the system. The cameras are arranged as described in Perfetti (2020) to provide a wide baseline between the optical centers, while keeping the footprint of the instrument small so that it is manageable even in tight areas. Ant3D is used in movement from a single operator at the normal speed of walk acquiring a sequence of timed and synchronized images. The acquisition of the images is controllable from the operator who can choose the frame rate with which to acquire the images automatically. Figure 4 shows an image of the multicamera device while Table 4 reports the main technical information of the system.
The field acquisition was carried out in October 2020 at the same time as the first acquisition carried out with Heron. The survey trajectory (Figure 1) starts at the top of the path, near Trona Grande, the first environment acquired (Figure 2, top). It then continues in a downward direction along the path and ends when the beginning of the route is reached near the village square of Prosto. Along the downward trajectory the two narrowest Trone were also measured (Figures 2, bottom). Table 3 reports details of the carried out trajectories. The acquired images were subsequently processed using the software Agisoft Metashape v1.7. Similarly to what was illustrated in Perfetti and Fassi (2022), initial values for the internal orientation parameters, previously obtained in the laboratory, were provided as input to the software and the calibrated distances between the cameras were imposed as constraints using the "scalebar" function of the software. Once the Structure from Motion (SfM) was finished and the correctness of the orientation of the images was visually checked, dense point clouds were produced with the same software. The point clouds obtained as output showed significant noise and it was therefore decided to filter the points by employing Metashape's confidence filtering, which filters the points based on the number of depth maps used to create them. Figure 5 shows a portion of the original point cloud, the applied filter, and the final point cloud following the removal of all points generated with depth maps < 4.    Table 4. Ant3D main specifications.

Positioning accuracy check
To evaluate the entity of the drift error of the two reconstructions, the coordinates measured by GNSS receiver (accuracy of about 5 cm) have been used as reference. For the verification it was decided to simulate a common scenario in the survey of confined environments: the two three-dimensional reconstructions were georeferenced using all the available references at the base of the path, in correspondence of the square of Prosto, while no GCP was positioned along the path. The drift error was then measured at three locations by calculating the difference between the estimated position of specific points on the ground and the control coordinate. As Figure 1 shows, check points were positioned at about 100m from the Prosto square, about halfway along the path, and at the end of the path, near Trona Grande. The point cloud produced by Heron Backpack was georeferenced during processing by using the already georeferenced TLS point cloud. The point cloud produced by Ant3D was instead georeferenced at the end of the SfM by performing a 7-parameter similarity transformation on the reference points available in the square. The total length of the trajectories covered by the two instruments was also calculated and consequently, the drift error of the two instruments for this specific application scenario, was estimated by dividing the maximum errors measured at the final point with the length covered by the trajectories.

Point cloud quality check
To evaluate the quality of the PC produced by the two survey methodologies, some specific areas of the PCs of the entire surveyed path were selected with the aim of quantifying the following parameters: (i) data completeness, (ii) acquisition range, and (iii) data noise. The areas selected for comparison are four ( Figure 1) and were chosen to represent a significant sample of the types of environments and surfaces surveyed in the case study in hand. In particular, the following areas were chosen: A1 "Trona Grande": it represents a large, confined environment (approximately 15m x 5m in cross section) used to compare data completeness and data noise.
A2 "Trona dell'Acqua": represents a confined environment of extremely narrow dimensions (about 3m x 1m of cross-section) that poses a challenge during the survey phase regarding the accessibility to the inside areas and regarding the manoeuvrability and portability of the instruments used. This area has been used to evaluate the completeness of the data and the noise of the PCs.
A3 "Undergrowth path": a section of the undergrowth path was chosen to evaluate the acquisition range, assessing in particular the extension of the PCs at the ground starting from the acquisition centre.
A4 "Flat surfaces": two portions of flat surfaces have been selected from the area of the square of Prosto, the vertical wall of the bell tower of the village and a portion of the paving of the square. For these portions it was decided to evaluate the point cloud noise.
The completeness of the data and the range have been evaluated by extracting cross-sections of the PCs for which it was possible to observe missing areas in the geometry of the caves and in the terrain in the undergrowth, allowing us to highlight the differences between the two systems. The noise of the data was instead evaluated using two distinct methods for the areas of the caves and for the flat surfaces. For the formers, the number of neighbors for each point within a radius of 20 cm was calculated from the PCs using the software CloudCompare. Having previously performed a subsampling of the PCs at a resolution of 5cm, the number of neighbors represents a measure of noise, highlighting a higher density in the areas in which the point clouds present more noise. On the other hand, for the latters a plane was interpolated on the subsampled data at 5cm and then the Cloud-to-Mesh (C2M) distance between the point cloud and the interpolating plane was calculated. The noise was then evaluated by comparing the histograms of the distances from the plane and evaluating the percentage of points falling within 3cm of the interpolated surface.

Positioning accuracy check
In Table 5, the drift error of the two systems checked on the 3 RTK GNSS points acquired along the trajectory is reported. For this specific application scenario, the trajectory drift can be then estimated by calculating the ratio between the error measured on the point farthest from the GCPs and the total length of the trajectory. In this way, Ant3D reported about 0.002 m of error per m of acquired trajectory. Heron Backpack reported about 0.02 m of error per m of acquired trajectory.

Point cloud quality check
In Figure 6, data density has been calculated for the two PCs for a section of the Trona Grande. The same has been performed for the Trona dell'Acqua ( Figure 7). In both cases, it is possible to notice that Heron Backpack PC reports a higher density in terms of n. of neighbors in a radius of 20 cm. Considering that the same 5 cm subsampling was performed for the two PCs, the obtained result indicates the noise produced by the Heron Backpack PC. Top of Figure 7 reports a comparison of the completeness of the PCs. It is evident how, in this very narrow space, Ant3D dimensions allowed to perform a better reconstruction of the environment. In Figure 8, a PCs comparison in a range of 30 m around the sensor is reported for a portion of the undergrowth path. In this case, also the Ant3D PC obtained without using the confidence filter is reported to demonstrate that, although the instrument acquires data in a fairly wide range, the reliable data are in a very limited range around the sensor. On the contrary, Heron Backpack PC is three-dimensionally correct and detailed.   Data noise check is then performed on a portion of the vertical wall of the bell tower ( Figure 9) and on a portion of the paving of Prosto's square ( Figure 10) by calculating the C2M distance between the PC and the correspondent fitting plane. By analysing the histograms of the C2M distances, the following percentages of points falling within 3 cm of the interpolated surface are here reported below in Table 6.
Regarding the Bell tower, the maximum obtained C2M distances are 1.29 m for Ant3D raw, 0.61 m for Ant3D filtered and 0.34 cm for Heron Backpack. As for the paving, they are 1.10 m for Ant3D raw, 0.07 m for Ant3D filtered and 0.10 m for Heron Backpack.

SYSTEMS INTEGRATION
In Chapter 3, the carried-out tests have pointed out the pros and cons of the products obtainable with the two systems. In this Chapter, the idea is to explore the possibility to obtain the best possible result by combining their strengths. This means creating the most accurate pc in terms of spatial positioning and, at the same time, capable of achieving information in a wide range from the sensor. The proposed experiment was performed using Heron Desktop software. Here it is possible to run the "Tracking Odometer" function, where a point cloud is imported and used as reference.
The trajectory is then reconstructed by forcing the IMMS LS data to adhere to the reference PC. At this purpose, the sparse point cloud coming from the photogrammetric processing of images acquired with Ant3D was imported and a reference map was then created by manually selecting regular points on the ground where the surveyor passed. This is necessary to tell the software which part of the reference PC is going to help the odometer in the appropriate trajectory reconstruction. The operator is assisted in the interpretation of the correct outcome of the process by the colours that the 32 lines of the IMMS LSs sensors take on: if they are green, data captured by LS and the reference PC match. Conversely, red indicates no correspondence between them. This may be due either to the actual lack of data in the reference PC (particularly evident in Figure 11-top), or to a lack of adhesion between the two PCs ( Figure 11-bottom). In this second case, the operator needs to stop the process, change the appropriate parameters, and resume the procedure. When the Backpack IMMS trajectory is reconstructed using the "Tracking Odometer" function, the associated point cloud can be directly exported to Reconstructor. Once exported, the accuracy check on the RTK GNSS points was performed, resulting in errors in the order of 10 cm. The sparse PC coming from the photogrammetric process of the multicamera system was chosen because it is made up of the most robust points and because it is the fastest point cloud resulting only from the pre-alignment of the images. The obtained result showed that, despite the limited range of points, it was sufficient to reconstruct the trajectory in a proper way. Figure 11. Trajectory reconstruction using the "Tracking Odometer" function in Heron Desktop software. Top: Right match between data on the ground, but lack of correspondence among the vertical elements. Bottom: Need to stop the procedure because of failure to adhere between points on the ground of Prosto's square.

CONCLUSION AND FUTURE WORKS
The obtained results clearly demonstrate the peculiarities of the two tested instruments. Ant3D achieved a positioning accuracy far better that the one obtainable with Heron Backpack, reaching a maximum error of 5 m in correspondence of the farthest point from the GCPs, whereas the Heron Backpack got 78.5 m.
Considering then the entire length of the trajectories, it can be concluded that the Ant3D drift per m is better by an order of magnitude than that of Heron Backpack (0.002 m and 0.02 m).
Regarding the quality of the produced PCs, it can be stated that Heron Backpack generated 3D data in a wider range, given the technical specifications of its embedded sensors. This results in a more complete reconstruction of the environment, when compared to the multicamera system one. When dealing with unreachable spaces, this has to be taken into account. On the other hand, the encumbrance and manoeuvrability of the handheld system make it more versatile in surveying very narrow spaces, as seen for the Trona dell'Acqua. Regard the noise of produced PCs, it is evident how Ant3D data need to be filtered to be reliable. Once selected the most confident points, the resulting PC is then characterized by very low noise if compared to the PC produced with Heron Backpack. The possibility to obtain an adequate unique final PC from both a quantitative and a qualitative point of view is successfully investigated, performing the trajectory reconstruction using the "Tracking odometer" function available in Heron Desktop. For the future, developing an instrument where the multicamera system oversees the positioning and the LS sensors are responsible for the production of the PC may be considered, especially when dealing with those kinds of wide spaces where it is not possible to pass over the same areas multiple times and therefore the SLAM algorithms are not helped. The ambitious next goal could involve the use of panoramic cameras, already integrated in the range-based system, to improve the positioning accuracy thanks to the trajectory reconstruction with the SfM process.