SENSOR INTEGRATION AND APPLICATION OF LOW-SIZED MOBILE MAPPING PLATFORM EQUIPPED WITH LIDAR, GPR AND PHOTOGRAMMETRIC SENSORS

: The article aims to analyse the possibilities of GPR, LiDAR, and photogrammetric sensors integration in a specific application, considering the various combination of sensor and their parameters. This text also discusses the possibility of using LiDAR sensors in a low-sized mobile platform for the inventory of the road lane in a dense urban area. The text presents the opportunities and recommendations of GPR and LiDAR sensors for their selection and the possibility of using them. In the case of LiDAR and photogrammetric data, two planned applications were indicated: platform georeferencing and mapping. The accuracy and noise of the Livox Avia LiDAR sensor and point cloud obtained from the Sony A7R camera with image-matching were analysed for a surface inventory. Despite the sufficient density and detail of the data, the intensity distinguishing different surfaces, the noise of LiDAR data at the level of 2 cm was too high to do the inventory of minor damages and analyse road surfaces. Higher accuracy was achieved at the level of 1 cm for photogrammetric point clouds. The article also presents the concept of integrating multi-source data visualised into the form of an oriented point cloud showing both what is above and below the earth's surface, which enables the synergy effect and joint analysis of data with entirely different characteristics.


INTRODUCTION
In recent years, the importance of monitoring, among others, road objects using non-invasive methods has increased. One of the main directions of development in this field is the combination of various non-invasive methods and advanced data analysis, including laser scanning (LiDAR) and ground penetrating radar (GPR) (Merkle et al., 2021) that allow for the effect of data synergy providing additional information in the result of the multisensorial data fusion.
The synergy of all these sensors on a single platform is provided for two main reasons: (1) Fusion of sensors for orientation purposes. GPR georeferencing using GNSS is optimal as long as measurements are performed in an open-sky environment (Barzaghi et al., 2016, Gabryś andOrtyl, 2020). However, the road infrastructure measurements and monitoring are often performed in GNSSchallenging environments (urban areas) or even indoor-like places (e.g. tunnels). In such cases, the GPR data orientation can be also based on photogrammetric vision systems (Barzaghi et al., 2016) or LiDAR data (Ogunniyi et al., 2020). LiDAR data can be also used for topographic correction within the sensor integration to provide higher accuracy of depth measurements from GPR (Merkle et al., 2021). * Corresponding author (2) Fusion of sensors for sensing purposes. These remote measurement techniques allow for the non-invasive measurement of large datasets about the space surrounding the measurement platform. The synergy of LiDAR, image-based, and GPR data will be desirable for diagnostic reasons. The data from the laser scanner and photogrammetric images allow the examination of the surroundings and the surface of the objecta road and its structure with a centimetre resolution, while the data from the ground penetrating radar allow the examination of road subsurface structure.
In this kind of sensor integration, in addition to the effect of linking what is above and below the road surface, it may be essential to look for correlations between what is below the ground surface and what is registered above by higher resolution remote sensing systems from the survey platform (photogrammetry and LiDAR).
Road surface monitoring is not the only application of integrated sensors. Integration of multi-source non-invasive techniques is increasingly being applied to building monitoring and management (El Masri and Rakha, 2020), in monitoring engineering structures such as bridges (Cafiso et al., 2019(Cafiso et al., , 2020, where autonomous platforms are also proposed (Merkle et al., 2020), roads (Li et al., 2020) and in various archaeological research (Kamp et al., 2014, Puente et al., 2018.
In the article, a prototype of a low-sized mobile mapping platform equipped with LiDAR, ground penetrating radar, and optical and navigation sensors will be considered to obtain the data in the selected application. Those sensors are reviewed and initially tested, referring to potential applications.

CONCEPTS
The synergy implemented in this article will enable mutual georeferencing of multi-source data. The developed methodology of joint data processing will allow obtaining information covered in data from multiple sources. The following section introduces some general decisions that have been made in the initial part of the project, referring to data collected with the designed prototype of a low-sized platform equipped with a Livox Avia LiDAR sensor, Sony A7R III camera, 2 Sony Alfa 6000 cameras, and with the opportunity to insert a preferable GPR antenna and GNSS receiver ( Figure 1).

LiDAR sensors in mobile platforms
Mobile laser scanning systems (MLS) have been used in road infrastructure monitoring for years. A meaningful change that has occurred in recent years in this field is the use of other sensors, where in addition to terrestrial scanners such as Faro or Riegl, lightweight systems used in navigation (Velodyne Puck, Livox) or unmanned laser scanning systems (ULS, e.g. Riegl miniVUX, LiAir) are more and more popular, allowing construction of smaller and cheaper platforms. Additionally, more and more lowcost solutions make the market more competitive, and LiDAR solutions are more available for more common solutions. Soilán et al. (2019), reassuming LiDAR-based Monitoring of Transport Infrastructures, mention the following applications: − Road Surface Monitoring (including road surface extraction, road markings, driving lanes, road cracks and manhole covers); − Off-Road Surface Monitoring (including traffic signs; pole-like objects roadside trees).
In the presented experiments, data from low-cost LiDAR Livox Avia were compared with terrestrial laser scanning data (TLS) acquired with Leica RTC 360 scanner. The data were collected stationary, and few ordinary objects were used for the data comparison. These objects were: cardboard boxes, spherical and round checkerboard TLS targets.
In Figure 2, data from two mentioned laser scanners were presented. The results show that the shapes of the scanned objects were reconstructed more appropriately on the TLS data. Livox point cloud was noisier, and it was a challenge to recognise used objects, even round checkerboards with contrasting colours. This may indicate that the LiDAR data accuracy from the Livox Avia scanner might not be enough for road inventory. The investigation of noise and roughness of LiDAR data for analysis of road planes, delivered with a less expensive scanner, should be done, which will be presented in the section with the results.

GPR antennas in mobile platform
The ground penetrating radar (GPR) method is a high-frequency electromagnetic technique used to produce high-resolution images. GPR is used, for example, for engineering, geological and archaeological investigations to detect specific objects in the ground (naturally occurring or man-made) or during road infrastructure investigations. Current applications of GPR include the location of buried objects, detection of voids and cavities, reinforcement location in concrete (He et al., 2009), geotechnical investigations, geological, environmental, archaeological, and hydrogeological surveys (Benedetto and Pajewski, 2015;Lejzerowicz et al., 2014). When investigating transportation infrastructure, GPR measurements include estimation of the thickness of the whole pavement package layer , detecting voids beneath layers (Lalagüe, 2010) and air void content estimation (Hoegha et al., 2015), locating reinforcement bars, inspecting pavement structure and mapping of underground utilities (Lalagüe, 2015;Lejzerowicz and Wutke, 2020).
For GPR, the maximal velocity that allows distinguishing items under the surface will be examined, which is related to the efficiency of data collection of the multisensory platform. During continuous profiling, GPR antennas are pulled/pushed along the profile. The number of scans per unit distance is a function of the pulse repetition and drag speed. It should be noted that the speed of pulling (or pushing the GPR on the measuring trolley) tends to change during profiling, which results from changes in the movement speed of the operator of the device. Therefore, we must consider the speed at which we can move using a given GPR antenna as the obtained results may differ from each other. This varies depending on the frequency of the used antenna, its resolution, or sampling speed, but all those parameters also depend on the manufacturer of the equipment.
An appropriate antenna should also be selected for a given task regarding the measurement frequency used. It all depends on whether we want to achieve a greater depth range of the measurement or its greater resolution. In the case of engineering research, e.g. when searching for underground infrastructure at shallow depths, we will use antennas with higher frequencies than, for example, during geological or geotechnical surveys where we are looking for objects located deeper below the land surface. In concrete tests, in searching for objects of small dimensions and at small depths and in the location of reinforcement, we use GPR antennas with frequencies in the range of 1-2.5 GHz. Such high-frequency antennas allow for a more precise location of a given object and find objects of small dimensions (the higher the frequency of the antenna used, the smaller the object you can locate because the vertical resolution of the measurement increases). However, it should be noted that with the increase in measurement accuracy and its greater resolution, a smaller depth range is associated. Therefore, it is unlikely to locate an object of the size of e.g. 2cm at a depth of several meters below the ground surface due to the lower resolution of the measurement due to the use of a lower frequency antenna. Figure 3 shows the same GPR profile obtained with two antennas with 300 and 700 MHz frequencies. This example shows the influence of the used antenna frequency on the measurement depth range and its vertical resolution: the 700 MHz antenna allows to obtain results with greater accuracy and locate a more significant number of objects (including smaller objects) below the land surface.

Integration idea
It is assumed that the platform with integrated sensors will be used in different conditions, i.e. outdoor and indoor. Indoor environments and locations between high buildings in cities do not allow for the possible use of the GNSS technique in georeferencing. It means that integrating multi-source data and IMU, visual, and LiDAR SLAM will enable georeferencing of the data when satellite navigation is unavailable. LiDAR and photogrammetric, oblique camera and side cameras may be used for point cloud generation of the road and its neighbour objects (building, facilities, vegetation). Their usefulness in mapping will be analysed in the result section.
Another critical issue in sensor integration is mounting on a platform. Such arrangement must consider the function of each sensor and its physical properties that require location on a platform. The example issue that must be considered is the minimum range of the laser scanner, which varies from 0,5 m up to 3 m for different scanners. In cities and more complicated building constructions, occluded areas may require more sophisticated platform route planning. The front oblique camera should be set with a proper angle to observe the road ahead and collect some areas on the roadside, which allows for further image orientation. GPR should be located as close to the ground as possible and far from other metal objects; GNSS sensors should be located high to fulfil the need for tracking position in dense urban conditions, while side cameras should be oriented to capture a view of the street passing by.
The most important for data integration is their synergy which is possible only when we have the same data type. For LiDAR and photogrammetry, such a product is a point cloud, and for GPR profiles in raster format. The idea for data fusion from low-sized multi-sensorial platforms was to present all data in one visualisation. Georeferenced radargrams were filtered and converted with anomalies that occurred. Next, they were converted for point clouds indicating objects under the surface that may be in the end-user's interest. In Figure 4a concept of LiDAR and GPR integration has been prepared. Anomalies in a transversal GPR profile indicates where some water supply water can be expected. Such a result can be confirmed by documentation or elements visible on intensity rasters, i.e., manhole hatches ( Figure 4b).

RESULTS
Test experiments were carried out with the platform equipped with a Livox Avia, Sony A7R III camera, 2 Sony Alfa 6000 cameras, GPR antenna model VIY3-300 and GNSS receiver Trimble R9s and controller Trimble TSC3. LiDAR data were collected using Leica RTC 360 terrestrial laser scanner (TLS). Test data were collected on the WUT university campus in front of the building and along the street.
Collected TLS data were processed in Leica Register360 software and imagesin RealityCapture. Livox Avia point cloud was acquired by combining LiDAR measurements and data from the built IMU. That process was conducted inside the ROS framework, using iterated extended Kalman filter. Some initial analyses of mapping sensors have been done in Cloud Compare.

Analysis of LiDAR data
Test data were collected with the Livox Avia scanner inside the Main Building of Warsaw University of Technology. The data were acquired inside the building to analyse the accuracy of the data in almost laboratory conditions. In Figure 5a, part of the data for the corridor floor with a regular pattern was presented. a) b) Figure 5. Test data acquired with Livox Avia sensor inside the building: with the noisy points (a) and with noisy points filtered out (b) Figure 6. Test data acquired with Livox Avia sensor inside the building with the noisy pointsfront view Analysis and visualisation of a double return point cloud, collected in non-repetitive circular scanning ("flower") pattern, in Figure 5a, a line of black points covering part of the tiled floor is presented. This means these points are characterised by very low intensity values. In Figure 6, the same point cloud was shown from the front view. The noisy points occurred only along the trajectory. These points are also registered higher than the other points measured on the floor. The noisy points may result from the number of measured returns of a laser beam. The experiments showed that only one laser beam return should be registered to monitor the road structures because more returns are unnecessary for road analysis, and more returns registered may result in a noisier point cloud. Elimination of those low-intensity points clearly showed the real pattern and plane of the floor surface ( Figure 5b).
To analyse the usefulness of the Livox Avia in road inventory, tests referring to noise and roughness analysis have been done. In Figure 7, the number of neighbours and roughness are shown. For this calculation, the noisy points were excluded from the study. The roughness of the point cloud was in the range from 0 to approx. 2 cm. The number of neighbours was mainly between 10 to 30 points in a 3 cm circular area, which means that a minimum of 1 point per 1 square centimetre was guaranteed in point cloud density, referring to the average human speed during data collection. Such results for the Livox Avia scanner mounted in a designed low-sized platform collecting data with walking human velocity provide data with proper density and intensity; however, the noise in case of flat areas is too large to use this sensor for damages monitoring. This type of LiDAR sensor could be supported in road inventory, but its role should be navigational in case of a low GNSS signal.

Analysis of photogrammetric data
Data from the platform were acquired in front of one of the Warsaw University of Technology buildings. This experiment focused on analysing photogrammetric data to investigate if they could be the primary mapping sensor in the designed platform. The images were acquired with a Sony A7R III camera, mounted in front of the platform and pointed at the pavement ahead (oblique view). The images were processed in RealityCapture software, dedicated to terrestrial photogrammetry applications. The images were oriented, and a dense point cloud was generated (Figure 8b).
For this experiment area, TLS data were also collected as reference data. The TLS data were processed in Leica Register360 software. Then in RealityCapture, TLS and image point clouds were merged (Figure 8a), and a 3D model was generated. a) b) Figure 8. Merged TLS and dense point cloud generated from images acquired with Sony A7R III camera for test area (a), a dense point cloud generated from images acquired with Sony A7R III camera (b) The dense point cloud from TLS was compared with the merged point cloud from TLS and acquired images. A TLS point cloud was treated as a reference. The point clouds were compared excluding trees, including only the pavement in front of the building. In Figure 9, Z-coordinate differences between points clouds: only from TLS and merged TLS and image point cloud.
The comparison results show that the differences between the point clouds are lower than 1 cm in total for the whole area. A compact camera can collect data that provides centimetre-level accuracy with a higher ground sampling distance than it is possible to produce for LiDAR intensity.

CONCLUSION AND FURTHER RESEARCH
The article aims to analyse the possibilities of GPR, LiDAR, cameras and navigational sensors integration in a specific application considering the various combination and their parameters in alternative to large, more expensive platforms that are difficult to maneuver. These applications of the platform might be: − inventory of parts of the road surface that need a detailed inspectionthe aim of such measurement is a thorough inspection of the condition of the surface and subsurface area simultaneously, − inventory of the foundations of industrial and storage halls, − inventory of the road surface before designing and installing new underground utilities in cities This article shows an example of a small mobile platform that could be used in hard-to-reach locations requiring mapping in dense urban areas. Selected sensors in a platform play both roles on the platform: navigational and mapping.
The concept of visualisation of multi-source data as oriented point clouds is interesting. In this paper, it was done with the help of reference data, but this data source can be successfully replaced by point clouds from compact side cameras using dense image matching. Due to poor results of the tested LiDAR unit in surface measurements suffered from high noise, it should be tested more in integration with the photogrammetric point cloud, used only for significant damages detection but mostly applied for navigational purposes. Future work under the implementation of LiDAR and visual SLAM algorithms should be investigated to navigate the platform for indoor mapping.