PORTABLE MULTI-CAMERA SYSTEM: FROM FAST TUNNEL MAPPING TO SEMI-AUTOMATIC SPACE DECOMPOSITION AND CROSS-SECTION EXTRACTION

: The paper outlines the first steps of a research project focused on the digitalization of underground tunnels for the mining industry. The aim is to solve the problem of rapidly, semi-automatically, efficiently, and reliably digitizing complex and meandering tunnels. A handheld multi-camera photogrammetric tool is used for the survey phase, which allows for the rapid acquisition of the image dataset needed to produce the 3D data. Moreover, since often, automatic, and fast acquisitions are not supported by easy-to-use tools to access and use the data at an operational level, a second aim of the research is to define a method able to arrange and organise the gathered data so that it would be easily accessible. The proposed approach is to compute the 3D skeleton of the surveyed environment by employing tools developed for the analysis of vascular networks in medical imagery. From the computed skeletonization of the underground tunnels, a method is proposed to automatically extrapolate valuable information such as cross-sections, decomposed portions of the tunnel, and the referenced images from the photogrammetric survey. The long-term research goal is to create an effective workflow, both at the hardware and software level, that can reduce computation times, process large amounts of data, and reduce dependency on high levels of experience.


INTRODUCTION
Over the years, 3D modelling from LiDAR (Light Detection and Ranging) and dense image matching has been an active field of research, now witnessing a flourishing development of mobile systems and technologies (Mobile Mapping Systems, MMSs) that have made point cloud and 3D data production more accessible and expanded the domain of application of 3D point clouds data. However, despite the availability of data, there is an underutilisation of its potential in practice, which may be due to several factors, including the size of the data, the difficulty of accessing significant parts individually, and the complexity of the evolving software and hardware ecosystem, which is often inaccessible to non-expert users. An example of this can be found in mining, the subject of this paper, which benefits from the development of MMSs and other portable survey devices for the time-effective acquisition of extensive tunnels. At the same time, the extension of the tunnels causes the 3D point cloud data to multiply in size, hindering accessibility and amplifying the challenges in extracting useful information, such as volumetric calculations, cross-sections, virtual inspections, etc.

Data acquisition in mining
In the last decade, technological advances in geomatics have changed the practice of surveying confined environments such as mines, tunnels, and galleries. Traditional instruments for discrete and punctual measurements, such as the total station, are now complemented by tools and techniques for a continuous survey, such as Terrestrial Laser Scanners (TLS) (Gikas, 2012;Cacciari * Corresponding author and Futai, 2015;Monsalve et al, 2019) and terrestrial photogrammetry (Charbonnier et al., 2013;Slaker et al., 2016;Attard et al., 2018;Panella et al., 2020). However, these acquisition techniques are not always ideal for surveying environments/tunnels that significantly extend in length. Indeed, they are the less effective, the more the section-to-length ratio of the tunnel decreases. For elongated spaces, in fact, the acquisition time needed to obtain a homogeneous data resolution increases, while instead, it should be minimised to avoid interference with vehicular traffic or existing mining activities. This is why the digitization of these environments is nowadays carried out with MMSs that allow acquisitions in motion and, therefore, shorter acquisition times. Vehicle-mounted MMSs have long been the standard for large tunnel acquisition in road and rail construction and maintenance (Boavida et al., 2012;Zhou et al., 2017) and mining (Zlot and Bosse, 2014a). In more recent years, more costeffective and wearable solutions, such as wearable MMSs, are being developed, tested (Bosse et al, 2012;Lauterbach et al., 2015), and employed for the digitization of complex environments, even on uneven terrain or in the presence of steps and obstacles where accessibility to wheeled vehicles is limited or impeded (Zlot and Bosse, 2014b;Farella, 2016;Brocchini at al., 2017, Fassi andPerfetti, 2019;Raval et al., 2019;Marotta et al., 2021). In addition, much less adopted, there are image-based approaches based on the acquisition of images or video sequences and SfM (Structure from Motion) and visual SLAM (Simultaneous Localisation And Mapping) algorithms. These exploit fisheye lenses (Troisi et al., 2017;Alessandri et a., 2019;Perfetti et al., 2020), panoramic cameras (Barazzetti et al., 2018;Perfetti et al. 2018) and multi-cameras (Meyer et al., 2020;Di Stefano et al., 2021;Perfetti and Fassi, 2022) for effective acquisition of narrow spaces. Continuous sensing systems, both range-based and image-based, have led to an increase in accuracy, resolution, and recordable detail, but at the same time, particularly the newer wearable MMSs and multi-camera systems, have led to a rise in the amount of data acquired as well.

Data requirements/usage in mining
Digital data such as the ones provided by TLS, MMSs, and image-based solutions, i.e., point cloud and images, are helpful in the field of mining at many stages of the activities and especially during the lifespan of the mine to perform periodic monitoring and inspections as part of the tunnel management process. Indeed, detecting any source of deterioration and defects at an early stage is a must to apply proper maintenance, extend the tunnel's life and ensure safety. Tunnel defects could take various forms, such as leakage, cracks, displacement, failure of structures, and more. Moreover, rock mass analysis is essential to identify overbreak and underbreak zones due to excavation. For these tasks, regular, automated, and cost-effective tools are required for continuous monitoring and sensing throughout the lifecycle of work. Visual inspection is probably the most valuable and desirable method for preliminary assessment of various features and signs of stress and instability. Hence, after detection, studies and analysis must be conducted to verify the existence of instability and eventually provide solutions. Cross-sections, point clouds, and surface models are examples of 3D data that can be utilised to extract punctual and periodic information as well as geometric properties such as volumes, spacing, distances, area, and orientation to perform various analyses on rock masses (Cacciari and Futai, 2015). In addition, images acquired through photogrammetric techniques or cameras equipped with TLS or MMSs provide effective tools for surface inspections such as cracks and defects detection (Attard et al., 2018).

Paper objectives
This paper has a twofold objective: (i), to validate a methodology to perform the 3D survey of tunnels and confined spaces, employing a laboratory prototype of a novel portable multicamera system; and (ii), to present a solution aimed at making the 3D data acquired by the proposed system accessible, i.e., a methodology and tool that can be used standalone or implemented in an online platform to empower the users to extract the data they need. The first part of this investigation focuses on two case studies, for which a complete 3D digitization has been carried out employing Ant3D multi-camera system. The aim is to evaluate the effectiveness of the proposed solution for the task of periodic mapping. In this regard, the test concentrates on assessing the practicality of the data acquisition operations and the accuracy, completeness, and quality of 3D reconstruction. The second part of this investigation focuses on the outputs of the 3D digitization, specifically on tests conducted with the purpose of organising the digital data and processing it to make it effectively available and usable in practice. A comparison is presented of two methodologies to subdivide the modelled spaces into their fundamental elements by automatically detecting forking and branching in the tunnel environment. The output of both methods is a collection of spline lines representing the centreline of the different detected branches, i.e., the skeleton of the whole 3D model. Subsequently, the semi-automatic retrieval of information is tested by exploiting the previously computed skeleton structure of the environment. Outputs such as cross-sections, segmented volumes, and the images acquired by the multi-camera system are retrieved from a combination of the source data (point clouds, models, and photos) and the skeleton structure of the 3D environment in hand.

CASE STUDIES
The investigation focuses on two case studies: the talc mine of IMI Fabi, Brusada Ponticelli (SO) and the fortification tunnels of Monte Celva (TN). The two areas have been selected to test the robustness of the multi-camera approach in two challenging scenarios.
IMI Fabi talc mine: The mine is still active in the extraction of the mineral and is distributed on 13 levels that intercept a vertical talc vein. The full extension of the excavated environments is around 20 km long; however, a lower extension of about 2 km has been surveyed for the present test. The object of the survey has been the main tunnel of access to the mine at the first level. It connects to an ascending helicoidal ramp that leads to all successive levels. The ramp was measured up to level 8, where the area of interest terminates in an open-air exit on the mountainside. The tunnel has a variable cross-sectional area of around 5 x 5 m. This case study poses two main difficulties: (i) the extent of the acquisition path, which is prone to highlight the drift error; and (ii) the fact that the surveyed path is an open path that does not reconnect with the starting point (Figures 1 and 2).
Monte Celva WWI fortifications: they are defensive structures forming part of the defence of the city of Trento, completed in 1915. Today the area consists of a series of tunnels and underground rooms dug into the natural rock and entrenched spaces at the top of the mountain made of masonry and concrete (Farella, 2016). The underground chambers are the focus of this test. They are narrow and tortuous, barely illuminated by the entrance openings and cover an area of about 45 x 45 m. Most of the tunnels are characterised by different sections from a maximum of around 3 x 3 m to approximately 1 x 1.8 m for the smallest ones. The main characteristics of the fortified complex, which motivated its choice as one of the test environments, are the complexity of the connections and the diversity of the connected environments ( Figure 7).

Proposed multi-camera system -Ant3D
The 3D digitization of the two case studies was performed using the laboratory prototype of a photogrammetric multi-camera called Ant3D ( In the field, the image acquisition is made automatically by selecting the acquisition frame rate while survey operations are conducted by a single operator at walking speed. In addition, the system can also be used autonomously mounted on a vehicle by controlling the acquisition remotely. Battery life allows for continuous acquisition of approximately 3 hours. The multicamera is designed for the 3D reconstruction of narrow spaces, with a 360° view and has been employed effectively in different scenarios, Perfetti and Fassi (2022) describe the use of Ant3D for the survey of architectonic narrow spaces in cultural heritage.

Data acquisition -IMI Fabi mining site
The acquisition was carried out in two days, on the first day the entire tunnel was acquired ( Figure 1) while on the second day some photogrammetric coded targets were placed and measured on the tunnel walls in the proximity of some vertices of the geodetic network existing in the mine (Figure 2). On this same second day, some additional acquisitions were performed with Ant3D in order to measure these targets. The main acquisition starts at the first level of the mine and continues along the entire upward path until it reaches the eighth level and then the exit on the mountainside. From here the acquisition covers the same trajectory in opposite direction until it returns to the starting point. In correspondence with the main crossroads, where several tunnels unravel from the same point, as it happens at the ramp landing at each new level, the acquisition has been reinforced by performing a ring path with the aim of robustly linking the outward path with the return one. In total, the acquisition took 3 h and 30 min to cover the whole route (3 h the first day and 30 min for the integrations) and a total of 22690 images (4538 multi-images) were acquired. The duration of the acquisition depended mainly on the walking velocity on the field, the acquisition could have been performed by mounting the multi-camera on a vehicle thus drastically reducing the acquisition time.
The whole image dataset acquired during the survey was processed in Agisoft Metashape (version 1.7). In the process, as described in Perfetti and Fassi (2022), the known baselines of the camera pairs were constrained using the scalebar function in the software. The result obtained from the SfM required a manual intervention phase in which some misalignments were fixed until the correct orientation of the images was obtained. At this point, the estimated values of the images' exterior orientation were extracted and used as approximate values in a second process. This allowed the recalculation of the SfM from scratch that yield correct results right away without the need of manual interventions.

Data acquisition -Monte Celva Fortifications
The 3D survey of the entire fortifications has been divided into multiple distinct acquisitions focused on different areas. The present paper focuses on just a portion of the second acquisition that covered the underground environment used in the further tests ( Figure 7). As with the IMI Fabi mine acquisition, the survey was performed on the first day in the field. On a second day, additional measurements were performed to survey some coded photogrammetric targets. The survey of the underground spaces took less than 30 min, and a total number of 5830 images (1166 multi-images) were acquired for the whole second acquisition. Data processing was done in two parts, in the same manner as for IMI Fabi.

Accuracy check
As described in Perfetti and Fassi (2022), the main evaluation of the accuracy of the 3D reconstruction, is that of the maximum drift error of open and unconstrained trajectories. Therefore, this verification was performed for the most challenging case study of the two, i.e., the IMI Fabi mine tunnel. The reconstruction was oriented by performing a 7-parameter transformation on 5 Ground Control Points (GCPs) at the beginning of the path. The drift error was then measured on Control Points (CPs) located along with the path extension ( Figure 2); Table 1 shows the errors obtained. The maximum error is ~1.8 m, obtained at the end of a path of about 2 km. Table 1 shows how the error grows progressively as the length of the unconstrained trajectory increases; overall, a drift error of ~9 cm per 100 m of surveyed path was obtained. Although the error achieved is about twice as large as that previously obtained in Perfetti and Fassi (2022) in an architectural application, it is still small considering the complexity of the path and the absence of constraints. Instead, by introducing control points distributed along the entire tunnel extension, the maximum error is reduced to about 10 cm, compatible with the accuracy of the coordinates of the photogrammetric targets.

3D reconstruction results and discussion
The survey carried out with the proposed multi-camera system has produced the desired results, proving effective in completing the digitization of complex spaces. Regarding the acquisition operation on the field, in the case of the IMI Fabi mine, it did not require the interruption of normal daily operations. The survey of the fortifications of Monte Celva took less than 30 min in an environment characterised by uneven terrain and narrow passages. In contrast, the survey of the mine, which lasted around 3 h, could have been further speeded up by completing the acquisition with a vehicle. The proposed solution is therefore effective in the field operations, ensuring fast acquisition times, and therefore employable regularly for periodic monitoring. The proposed solution is also effective regarding the quality and completeness of the output. The geometric data obtained, such as point clouds and mesh models (Figure 4)

PROPOSED APPROACH FOR DATA QUERY
The proposed workflow introduces a method that allows arranging and accessing the data by means of a 3D skeleton representation of the surveyed environment that can be extracted from the survey data alone. Such structure allows easy and efficient extraction of cross-sections and of individual tunnel branches or portions of branches, i.e., decomposition of the tunnel environment; and to perform various geometric measurements and to reference thousands of scattered unarranged images. Figure 6 summarises the workflow.

Skeletonization
The skeleton provides useful information on the geometry and topology of an object. It serves as a shape descriptor representing a simplified version of the object but retrieving all its geometric features. It allows pinpointing specific regions of interest to assess the original data of only that region. Specifically for the data gathered by Ant3D, and similarly for MMSs, a simpler approach exists that would produce a similar output to the skeleton representation, which can be used successfully to a certain extent. That is the retrieval of the survey trajectory followed by the instrument during acquisition that can be obtained by interpolating a spline line on the estimated exterior orientations of the oriented cameras. Provided that individual survey sequences are continuous, i.e., every image in the sequence is in proximity to the previous and next ones, this approach would result in a collection of trajectories that describes the surveyed environment. Nonetheless, this procedure has several downsides, such as (i) the non-centeredness of the trajectory, (ii) the discontinuities due to interrupted acquisition sequences, and (iii) the overlap of different trajectories in the same area, as for the outward and return acquisitions. In other words, the survey trajectory is not topologically synchronised to the environment's features.
For this reason, skeletonization provides a more efficient shape descriptor since the 3D skeleton represents a topologically synchronised simplification of the modelled environment composed of a collection of spline lines, each representing the centreline of the tunnels. Several skeletonization techniques exist: 3D data can be skeletonised by either 2D-or 3D-based methods. 2D skeletonization proposes the conversion of 3D point clouds into 2D planar binary images and accordingly estimates the centreline on the 2D grid (Han et al., 2011). This fails to interpret tunnel models with complex 3D geometry, for example, tunnels with spiral shapes where multiple levels overlap. The bottom levels are covered by upper levels, making them invisible. Therefore, they require decomposition of intersecting trajectories and later merging after skeletonization. This leaves 3D skeletonization techniques as the most efficient method to implement.

CSD:
The Cylindrical Shape Decomposition (CSD) is a fully automated algorithm that partitions objects into a combination of semantic components (Abdollahzadeh et al., 2021). It provides both skeletonization of objects and cylindrical decomposition. However, in this study, the main interest is to test the first part of the algorithm that allows the computation of the skeleton curve. The CSD was developed to process 3D biomedical imagery in order to analyse vascular networks and blood vessels, but, as the authors themselves show, it can be applied to other domains of similar structure, such as, in our case, mining tunnels. As proposed in Abdollahzadeh et al. (2021), the skeletonising approach initiates by determining a point inside the object's domain having the furthest distance to the object's surface. Then, a second point is chosen inside the object, at the furthest geodesic location from the previous. A curve is traced connecting the two points along the tubular environment between the two points. A cost function is defined to ensure the centeredness of the path. After the first branch is traced, the location of the second point is updated to be positioned at the furthest geodesic point from the current state of the skeleton, and it becomes the starting point of a new branch connecting to the previous one. The process is repeated until the 3D skeleton is completed ( Figure 5). The CSD works on 3D voxel objects. Therefore, the point cloud had to be transformed into a binary volume representing the voxel object, with values of 1 representing the inside of the tunnels and values of 0 representing the empty space. By controlling voxel resolution, more peripheral branches can be detected. The voxel object representing the tunnel is then supplied as an input to the algorithm. The output of the CSD is a list of coordinates of points belonging to the skeleton with ID values provided for each of the branches. The 3D skeleton representation is constructed by interpolating the obtained list ( Figure 5). VMTK: The Vascular Modelling Toolkit (VMTK) is a library providing several functions to handle and model vascular structures. Conceptually, a vascular system and a tangle of underground tunnels are similar in shape, complexity, and irregularity. For this reason, the use of the vmtkcenterlines function (Antiga and Manini, 2013) could provide good results to model underground mines and compute the tunnel centrelines. The algorithm functions on the concept of maximal inscribed spheres centred at every point belonging to the Voronoi diagram of the 3D model. Once the radiuses have been determined, a wave is propagated from a source point, known as the centreline's starting point, measuring the wave's arrival time at all points on the Voronoi diagram. Then, the shortest path, which minimises the integral of the radius maximal inscribed sphere along a path, is defined as the centreline (Antiga, 2002). Compared to CSD, VMTK takes surface models as inputs and imposes the selection of source points which act as start and endpoints of centrelines to be computed. Therefore, manual involvement is required to initiate the process. The algorithm's output is like the CSD one; a list of points belonging to the skeleton is provided with associated IDs representing the branches. The 3D skeleton is then computed by spline interpolation of the given points (Figure 7).
Both the CSD and VMTK algorithms were used to extract the 3D skeleton of both the IMI Fabi and Monte Celva models. The resulted outputs ( Figure 5 and 7) have been tested to evaluate their accuracy and efficiency to serve as shape descriptors. Specifically, since the centeredness of the computed spline is essential for the accuracy of cross-section extraction and crosssection orientation, a check was carried out to assess the error in the spline centeredness with respect to the centre of mass of the computed cross-section. In the case of the IMI Fabi skeletons, 20 cross-sections were computed for each of the two approaches by obtaining the 3D coordinates of 20 points from the spline lines and their associated normal vector orientated along the spline direction. Then, the horizontal and vertical accuracy of the spline lines has been assessed as the Root Mean Square Error (RMSE) between the coordinates of the 3D points extracted and the coordinates of the centres of mass computed from the crosssections. The RMSE of the horizontal and vertical accuracy was 0.16 cm and 0.28 cm for the VMTK compared to 0.86 cm and 0.2 cm for the CSD, respectively. The low accuracy obtained for some of the points reflects the high perturbation of the tunnel geometry that deviates from a circular shape. Nonetheless, both methods efficiently extracted splines even on diverse shaped models of multiple levels and complex interconnected passages.

Cross-section extraction
Cross-sections play an essential role in capturing the geometry of the tunnel, as they are analysed to study the structure's dimensions, deflections, deformation, and eventually stability. Traditional methods rely on sparsely sampled points acquired by total stations which are time-consuming due to the nature of tunnels spreading over vast areas and are error prone. With the availability of dense data provided by 3D surveying, it is possible to extract detailed cross-sections nearly at any preferred location. The method proposed is to slice the volumetric data set along with a plane having a normal vector parallel to skeleton spline line, producing an orthogonal slicing plane. The skeleton spline must be smoothed to avoid any sharp turn in the curve caused by tunnel irregularities and thus assuring that the slicing plane is properly perpendicular to the tunnel direction. Cross-sections can now be extracted automatically at any defined location, interval, and number by selecting a skeleton branch and a position along the spline. Figure 8 shows some examples.

Space decomposition
Tunnel decomposition is essential when dealing with tunnels expanding over several kilometres and vast areas. Dealing with a complete tunnel model is time-consuming and hardware demanding. Taking advantage of the obtained skeleton decomposed into sub-skeletons each labelled with a specific ID, we can extract and isolate any preferred branch automatically or manually. For example, clipping boxes can be manually positioned at defined point coordinates of the spline skeleton to extract and isolate portion of the tunnel. Furthermore, the VMTK library can decompose the tunnel based on the sub-skeletons computed automatically without any manual intervention (Figures 7 and 9).

Images query
Images provide an effective tool to monitor and inspect tunnel environments compared to traditional methods that required the physical presence of trained personnel. Using images combined with image processing algorithms, surface properties such as cracks and water leakages can be automatically detected.
For the IMI Fabi mine, a total of 22690 images were acquired during the survey operations, for a total size of 33 gigabytes. Due to the number and size of the images, manual navigation at preferred locations to perform monitoring and inspection seems unrealistic.
The proposed solution was to automatically reference images along the skeleton to localize each image in its proper position in space. Given the 3D coordinate of both the images' exterior orientation and the 3D skeleton, a link can be built between similar corresponding coordinates. Since the multi-camera system which was used during the survey is composed of five cameras, then, every multi-camera position is defined by the centre of mass of the estimated coordinates of five images. A threshold is then defined which represents a radius limit to be considered around each point selected from the skeleton, then, all multi-image following within the range can be accessed and displayed. If a point of interest is defined on the skeleton, then, all images insisting on that location can be promptly accessed ( Figure 10). Figure 10. Example of the fisheye images extracted from the photogrammetric survey.

CONCLUSIONS AND FUTURE WORKS
The mining industry has benefitted from the rapid development of 3D surveying technologies that increased the offering of fast and detailed mapping solutions able to produce quality 3D data to be utilized to perform various crucial tasks in the highly demanding environment of underground tunnels. Furthermore, automatic extraction tools allow mine professionals to display and manage this vast amount of 3D data according to their needs throughout the mine's lifespan.
In this paper, we presented a test aimed at evaluating the effectiveness of a multi-camera system solution (Ant3D) for the rapid survey of underground environment. The proposed system proved to be effective in both the acquisition time on the field and in the quality of the output 3D data. Indeed, the acquisition phase never required interrupting the mining activities in progress and the resulted point cloud and surface model were complete in all their parts. form the data gathered, A test was performed to extract the 3D skeleton representation of the modelled tunnel environments by employing two algorithms designed for vascular networks analysis. Both algorithms proved effective in retrieving an accurate simplified shape representation of the tunnels and the computed skeleton was successfully used to automatically extract cross-sections, portion of the source data, and the images acquired during the photogrammetric survey; all by querying specific locations along the skeleton spline curve. The proposed approach for data access through the skeleton representation of the environment is a simple and effective tool to manage large amounts of data and retrieve the needed information on demand. Future works could concentrate in implementing the tested approach on an open-web 3D platform such as CESIUM that allows to share and access 3D data online. The skeleton structure can therefore be implemented as an organisation and extraction tool that can enhance the possibility of visualising and using the source data.