ON PRACTICAL ACCURACY ASPECTS OF UNMANNED AERIAL VEHICLES EQUIPPED WITH SURVEY GRADE LASER SCANNERS

: Unmanned aerial vehicles (UAVs) equipped with image sensors, which have been widely used in various fields such as construction, agriculture, and disaster management, can obtain images at the millimeter to decimeter scale. Useful tools that produce realistic surface models using 3D reconstruction software based on computer vision technologies are generally used to produce datasets from acquired images using UAVs. However, it is difficult to obtain the feature points from surfaces with limited texture, such as new asphalt or concrete, or detect the ground in areas such as forests, which are commonly concealed by vegetation. A promising method to address such issues is the use of UAV-equipped laser scanners. Recently, low and high performance products that use direct georeferencing devices integrated with laser scanners have been available. Moreover, there have been numerous reports regarding the various applications of UAVs equipped with laser scanners; however, these reports only discuss UAVs as measuring devices. Therefore, to understand the functioning of UAVs equipped with laser scanners, we investigated the theoretical accuracy of the survey grade laser scanner unit from the viewpoint of photogrammetry. We evaluated the performance of the VUX-1HA laser scanner equipped on a Skymatix X-LS1 UAV at a construction site. We presented the theoretical values obtained using the observation equations and results of the accuracy aspects of the acquired data in terms of height.


INTRODUCTION
The acquisition of spatial information using unmanned aerial vehicles (UAVs) is generally utilized in various fields such as 3D modeling of a complex structure using UAV images and terrestrial images for archaeology (Nocerino et al., 2013), crop surface modeling for agriculture (Possoch et al., 2016), low cost canopy height modeling for forest monitoring (Mlambo et al., 2017), and disaster analysis using ortho-images and DSMs with a spatial resolution of 0.06 m (Saito et al., 2018). Realistic textured surface models can easily be created using images obtained from UAVs and 3D reconstruction software. However, there are some issues with redundancy in the data size of overlapping images and processing time, and it is difficult to obtain feature points from images with poor textural information or ground surfaces in forested areas. UAVs have therefore been equipped with measuring systems that consist of georeferencing devices (i.e., GNSS/IMU) and laser scanners. Due to the varieties of georeferencing devices and laser scanners, they can be classified into a range of low to high performance products. The Velodyne VLP-16 laser scanner is often adopted in low-cost measuring units and found in the collision avoidance laser scanners used in the automotive industry. Jozkow et al. (2016) demonstrated the use of topographic mapping using sensors by integrating the Velodyne VLP-16 with the NovAtel GPS and the Epson IMU. An average distance of 0.90 m was calculated by CloudCompare using the point clouds collected from the laser scanner sensors and point clouds produced by the SfM (Structure from Motion) and MVS (Multi-view Stereo) using a Nikon D800 * Corresponding author camera with a Nikkor AF-S 50 mm f/1.4G lens. The authors carried out a performance evaluation of the Velodyne VLP-16 with respect to both the geometric and radiometric aspects of the acquired data (Nakano et al., 2018). The results from the evaluation of the height accuracy revealed an RMSE of ±0.10 m, including the point clouds where no adjustment was made using ground control points. The error was further reduced to approximately ±0.04 m by using nine ground control points. These results were reasonable; however, no comparison with the theoretical values was carried out (the values expected according to the sensor configuration and fight conditions). Pilarska et al. (2016) reviewed the potential of light-weight laser scanners by analyzing both the sources of errors and influence of such errors on the measurement accuracy. Sofonia et al. (2019) produced detailed results which indicated that flight plan parameters affected the measurement when the UAV laser scanner system, "Hovermap" by SLAM, was used without the GNSS. However, the article did not include evaluations of direct georeferencing in the normal or emergency mapping considered in this study. Recently, the performance of UAVs has improved because of the development of modern technology. This has resulted in reliable and high-powered UAVs that can lift 10 kg payloads, including devices used for agrochemical spraying. It is therefore possible to realize highly accurate measurements using heavy and highaccuracy survey grade devices. Several survey grade laser scanner units for UAVs are available. The products produced by LiDAR USA that combine the GNSS/IMU with Riegl VUX-1HA laser scanner, which was built as a high-accuracy measurement system similar to that of mobile mapping systems, were adopted in this study to evaluate the practical performance of UAVs equipped with survey grade laser scanners. Theoretical values were also calculated to serve as an evaluation index for the observation equations used in the calculation of 3D points using error propagation. The data was acquired from a construction site and the accuracy was assessed via comparison with the 3D reconstructions using images and terrestrial laser scanners. Pilarska et al. (2016) used the mathematical model based on an error model proposed by Vosselman and Maas (2010) to analyze the components of ranging, angular, and GNSS/INS positioning and orientation errors without reference to bore-sight errors or lever-arm errors. It was found that the influence on the lever-arm errors was negligible when the calibration was carried out correctly. The bore-sight calibration was said to be important and UAV lever arms were smaller than the airborne laser scanners; however, small errors in the bore-sight calibration cannot be ignored in any study concerning high accuracy measurement. The evaluation index was calculated by considering the theoretical values derived from the formula describing the 3D coordinates of the measurement points. The formula for the theoretical value of the measurement point produced by the UAV equipped with laser scanner was derived from the equation proposed by Skaloud and Lichti (2006), in which the importance of the bore-sight self-calibration in an airborne laser scanner was examined. ) is the skewed symmetric part of the boresight matrix, with small angles α, β, and γ; * as an a priori known rotation matrix from the laser frame to the IMU body frame that depends on a particular system mount; ρ and Δρ describe the laser range at time t and the constant range-finder offset, respectively; θ is the angular value of the laser encoder at time t (an offset of Δθ or other parameters that influence the beam direction can be added); and ⃗ = [ ] is the lever-arm offset between the IMU and laser measurement centers expressed in the frame of the IMU body. Equation (1) takes the matrix Ω considering the bore-sight selfcalibration b, the rotation matrix T of the laser scanner with respect to the IMU body, the position of the IMU from GNSS in the target coordinate system m, the attitude R from the IMU, the distance measurement ρ, the value used to correct the distance measurement Δρ, the scan angle θ, and the lever arm into account. A value of approximately 0.10 m was calculated as Δρ for the bore-sight self-calibration, reducing the variation in the point cloud. However, the distance resolution of the airborne laser scanners was approximately 0.05 m in 2006. It is therefore assumed that Δρ has limited effect on the current laser scanner specifications when measuring distance. Equation (1) was therefore simplified to equation (2) in order to obtain the observation equation used in this study.

DERIVATION OF FORMULA FOR THEORETICAL VALUES
(2) where [ ] are the coordinates of the measuring point; [ ] are the coordinates at the center of the IMU; = ( , , ) is the rotation matrix from the IMU; is the skewed symmetric part of the boresight matrix; is the known a priori rotation matrix from the laser frame to the IMU; ρ is the measuring distance; θ is the scan angular value; and [ ] is the lever-arm.
Error propagation was then applied to the observation equation in Equation (2), and the variance in the point of measurement was theoretically determined using the partial differential coefficient and variance (i.e., standard deviation) in each parameter. Equation (3) was used to calculate the theoretical value for x at the measurement point. The theoretical values and were also calculated using the partial differential coefficients. (3) where σ is the mean standard deviation of ; and are for the position of the laser scanner unit; , , and are for the attitude angle; is for the measuring distance; is for the scan angle, , , and are for the boresight angle, and , , and are for the lever arm.

EQUIPMENT USED FOR THE EVALUATION
The measurement equipment used in this study was the SkymatiX X-LS1 multi-copter UAV and LiDAR USA laser scanner unit. Figure 1 shows a UAV equipped with laser scanner unit. The X-LS1 can fly continually for approximately 10 min with sufficient motion performance despite carrying a laser scanner unit that weighs approximately 8 kg.
Although the SkymatiX X-LS1 is relatively large, with a rotor shaft length of approximately 1.5 m, it does not require a large space for transport and storage because the integrated rotor and frame can be detached from the main unit. The DJI-A3 is used to control the flight of the UAV according to the planned measurement schedule, and can be monitored using a hand monitor. The specifications of the laser scanner unit shown in Table 1 indicate that the AP40 of GNSS/IMU measures position and orientation at high spatial resolutions, while the RIEGL VUX-1HA, a survey grade laser scanner, measures the relative distance from the main unit to the ground with an accuracy of 5 mm.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2020, 2020 XXIV ISPRS Congress (2020 edition) Figure 1. Skymatix LS-1 UAV Table 1. Laser scanner specifications The ranges of the pulse repetition and scan rate in Table 1 were derived from the trade-off relationship between the pulse repetition rates, scan rate, and scan angle (i.e., irradiation interval) of the laser scanner. A similar trade-off between the maximum measuring range and pulse repetition rate occurs because the high-frequency pulse repetition rates reduce the effective measuring range more as opposed to the low-frequency pulse repetition rates.

DATA ACQUISITION
The data used in the performance evaluation was obtained using a laser scanner, digital camera, and terrestrial laser scanner. The coordinates of the test target was observed using GNSS and Total Station.

Study area and observation of the test target
The study area was a nearly flat construction site consisting of both earth and sand, with a total area of approximately 200 m × 100 m. The area included an approximately 2.5 m mound, which can be seen in Figure 2. The surface of the study area was marred by wheel tracks of heavy vehicles, puddles, and streams. The area delineated by the broken black line in Figure 2 is the verification area used for performing a spatial evaluation of the point cloud over a range of 50 m × 50 m. The coordinates of the two reference points were acquired using the GNSS static observation, and the coordinates of the 25 test targets with control and check points indicated by blue and orange circles, respectively, in Figure 2, were observed via a radiation method using Total Station.

UAV Laser data acquisition
The acquisition of the point cloud was carried out with a UAV equipped with a laser scanner. The process was performed three times using two settings to evaluate the stability of the equipment and reliability of the measurement. The two settings used for measurement are given in Table 2. The flight course was repeated three times for each setting, with a 50% side lap.   Figure 3 is a tint map that was overlaid with laser reflection intensity such that the blue, green, yellow, and red colors represent intervals of 5 m for the data acquired up to 100 m as specified in setting 1. Comparing Figure 3 with the ortho-image in Figure 2, trapezoidal mounds were evidently observed near the center of the upper part of the figure, and a white area representing the missing data can be seen in several puddles. The wind speed on the ground was approximately 2.0 m/sec at the time of data acquisition, but the surrounding trees were swaying as a result of breeze, suggesting that stronger winds were occurring at greater heights. The processing of the laser scanner data was carried out using the ScanLook PC, an application of the LiDAR USA. The integrated analyses were performed after the flight trajectory of the GNSS/IMU using Applanix POSPac, and the distance data from the RIEGL VUX-1HA were analyzed. The point cloud was adjusted using five control points determined by Terrasolid TerraMatch in order to improve the consistency with the ground coordinates.

Reference data acquisition
4.3.1 3D reconstruction using SfM/MVS: A 3D reconstruction of the study area was carried out using the digital camera images for i-Construction, which was developed by the Hazama Ando Corporation using Agisoft Metashape. This procedure was designed to satisfy the Japanese regulations for public surveying and construction surveys using UAVs. In the image acquisition specifications of 1 cm for the ground sample distance, a 90 % overlap and 60 % side lap were used. The area covered by the image acquisition was enlarged to cover the entire measurement area with the laser scanner. Seven test targets were used as ground control points in order to satisfy the regulation covering the arrangement of 3D reconstructions. A total of 149 images were acquired using a Sony α7 mounted with a 28 mm lens, 45 m above the ground.

Terrestrial laser scanner:
The reference point cloud was measured using the terrestrial laser scanner, Leica Geosystems ScanStation P50 (ranging accuracy 1.2 mm + 10 ppm), to map the verification area outlined by the black broken line in Figure 2. Because the verification area was approximately flat, the installation intervals were narrowed with respect to the angle of incidence of the terrestrial laser, and measurements were taken from ten stations. The point cloud of the terrestrial laser scanner exceeded 600,000 points/m 2 around the station; however, the average point cloud density was approximately 30,000 points/m 2 in the verification area.

PERFORMANCE EVALUATION
The accuracy was quantitatively evaluated by comparing the test targets to the two types of reference points; a point cloud based on a 3D reconstruction using SfM/MVS and point cloud based on a terrestrial laser scanner.

Evaluation using coordinates of test targets
The accuracy of the point cloud data obtained from the laser scanner was quantitatively evaluated using test targets. It was difficult to accurately recognize the shape of the test target from a point cloud that was obtained via discretization. The accuracy was therefore verified using a height interpolated from the coordinates of a neighboring point cloud to the planimetric coordinates of the test target. The most probable value for height was calculated using the Inverse Distance Weighted method, including all point cloud data within a radius of 0.09 m around the planimetric coordinates of the test target.
The following five items were evaluated with a total of 25 points in the form of five control points and 20 check points; the RMSE using the most probable value, variance from each point from the test target, range (maximum difference calculated from the maximum and minimum values for the coordinates of each test target), average (trend of the residual of the most probable value), and maximum residual of the most probable value.
From the results obtained at 100 m and 75 m, which are depicted in Figures 4 and 5, it can be confirmed that the RMSE and variance were less than ±0.02 m, maximum residual was less than approximately ±0.04 m, and range of variation in the point cloud was approximately 0.07 m.

Evaluation using reference results
The reference point cloud was evaluated as described in the previous section. The evaluation of the 3D reconstruction of the point cloud resulted in an RMSE, variance, range, average, and maximum value of ±0.025 m, ±0.026 m, 0.031 m, -0.020 m, and 0.050 m, respectively. The accuracy of the i-Construction was confirmed for this reason. The test target posts were filtered out of the terrestrial laser point cloud in accordance with the angle of incidence. The evaluation of the point cloud obtained from the terrestrial laser scanner resulted in an RMSE, variance, range, average, and maximum value of ±0.014 m, ±0.015 m, 0.037, -0.010 m, and 0.026 m, respectively, from the 13 test targets in the verification area. Figure 6 shows the verification area in both natural color and tinted map with the laser reflection intensity overlaid. The moist soils of the streams and puddles in Figure 6 were dark. Figures 7  and 8 show the results of the difference between the height from the 3D reconstruction point cloud and terrestrial laser point cloud in the verification area using the dataset 1 in CloudCompare. It can be seen from the histograms in Figure 7 and 8 that most of the differences were within ± 0.04 m. There are greater numbers of red dots that represent vegetation in Figure 7, which have been filtered during the 3D reconstruction, and the differences in the UAV laser point cloud were high. The differences can be seen in the wheel tracks in Figure 8; however, it was presumed that the UAV with laser scanner measured the shaded area produced by the terrestrial laser scanner by using the difference in irradiation angle.

DISCUSSION
The results of the performance evaluation confirmed that the UAV equipped with a survey grade laser scanner exhibited the highest accuracy and stability. The RMSE and variance in the height using the coordinates of the test targets in the point clouds, which were obtained from three repetitions with two settings, were less than ±0.02 m, and the difference between the two reference point clouds was within ±0.04 m.
We confirmed the validity of the evaluation by comparing the obtained results with the theoretical values. The theoretical values were 75 m and 100 m, corresponding to the setting, scan angle of 0° nadir, and course edges at 30° and 40°. The flight and scan directions matched the X and Y-axes, respectively. The standard deviation was ±0.02 m, depicted by σX, σY, and σZ, of the position obtained via GNSS/IMU, because the study area was suitable for receiving GNSS signals. The boresight angle and lever arm were set to approximate values of ±0.001° and ± 0.005 m, respectively  Table 3. Theoretical values Figure 9 shows the results of the calculation obtained by changing the flight height from 25 m to 150 m. It confirmed that the scan angle had a greater effect in the horizontal value than that of the change in theoretical height. Moreover, the theoretical horizontal value increased with increasing height of flight. It can be confirmed that the white patches in Figures 3, 7, and 8 are blank areas and lines caused by missing information. The blank areas and missing lines were due to the presence of puddles and sparse measurements caused by pitching, respectively. The near-infrared laser light has the property of being absorbed or reflected by water. Because it was raining before and on the day out of measurement, the conditions were not perfect. Although the missing data did not directly affect the geometric accuracy, it affected the interpretation of the point clouds. Therefore, we need The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B1-2020, 2020XXIV ISPRS Congress (2020 to consider ways to supplement the data acquisition, such as planning for multiple measurement periods.

CONCLUSIONS
A practical performance evaluation of a UAV equipped with a survey grade laser scanner was conducted in this study. The theoretical values were derived from the calculation formula for the 3D coordinates of the measurement points using the laser scanner. The data was obtained from a construction site using a UAV equipped with a laser scanner, images for 3D reconstruction, and terrestrial laser scanner. The five quantitative height assessments using test targets were performed using the point cloud that was produced by the UAV equipped with laser scanner. These were then compared to those produced from the 3D reconstruction and terrestrial laser scanners. The results revealed an RMSE of ±0.02 m for the most probable value for the height using the point clouds with adjustment. The difference between the point cloud of the UAV, equipped with laser scanners with 3D reconstruction, and the terrestrial laser scanner was approximately ±0.04 m. The evaluation results of the RMSE using the point cloud from the UAV, equipped with laser scanner, and theoretical value were equivalent at ±0.02 m. It was therefore concluded that the point cloud produced by the UAV equipped with a survey grade laser scanner has the ability to produce high accuracy measurements in the order of centimeters, in terms of height. As a result, the acquisition of spatial information using UAVs equipped with survey grade laser scanners was found to be applicable in the management of construction sites. However, some issues require further investigation. The horizontal accuracy of the point cloud was not evaluated in this study. Therefore, we aim to continue with this research and develop a better understanding of the accuracy of UAVs equipped with laser scanners.