Rapid Disaster Mapping through Data Integration from UAVs and Multi-sensors Mounted on Investigation Platforms of NDMI , Korea

As the recent disaster is so difficult to predict when and where it would hit, so it requires paradigm for disaster shifts from response to preparedness. In order to respond this change, NDMI has studied disaster scientific investigation (DSI) technologies for revealing systematically the root cause of disaster and protecting repetitive recurrence. The purpose of this study is to propose a convergence approach between data acquired from different types of sensors on a van-type investigation platform and UAVs of NDMI and assess their applicability for timely natural and man-made disaster mapping and monitoring. In order to evaluate its applicability for rapid disaster mapping, we pre-tested the proposed approach for NDMI site in Ulsan, Korea. For the enhancement of the direct geo-referencing accuracy of UAV imagery captured from on-board camera of DJI and the creation of more accurate map products, camera IOPs refinement and bundle adjustment were also performed with minimal GCPs. Finally, we conducted UAV data registration with LiDAR point clould for disaster mapping applications.


INTRODUCTION
Recent natural disaster has occurred in the different and unexpected aspects unlike the past.Economic and social losses and casualties caused by natural disasters have been increased every year in Korea.The amounts of damage are estimated at about $ 400 million each year over the past decade.According to these challenges as considered, post-disaster investigation paradigm has been changed from response to preparedness.It is because not only it is impossible to determine the exact cause of disaster with the existing technical limitation but also natural hazard events can trigger chemical and technical accidents.In order to cope with unexpected disasters, multi-sensors based data integration analysis and strategic approach are needed.
Korea's National Disaster Management research Institute (NDMI), responsible for implementing R&D related to national disaster and safety management, has studied for disaster scientific investigation (DSI) since 2013.DSI, a highly organized framework to reveal the root cause of disasters, aims to implement, monitor and feedback with disaster profiling and state-of-the-art forensic technologies.For DSI, NDMI has started to operate the various types of investigation platform and devices: a MMS-type specialized vehicle, UAVs, ultrasonography detector, rebar detector, etc.In recent years, Unmanned Aerial Vehicle (UAV) with various on-board sensors is considered to be a cost-effective tool for large scale aerial mapping.Its suitability to mapping applications is dependent on mapping extent, geometric accuracy, the durability such as flight time and control distance, and the heavy lifting capacity of UAV.It is equipped with the GNSS/IMU, MEMS, gyroscopes, accelerometers, and barometer to conduct direct sensor orientation.Precise timetagging of the camera shutter and GNSS time enable it to annotate the position and attitude information on the metadata of captured imagery (Rehak et al., 2013).Direct geo-referencing of airborne sensor which is an alternative to AT (Aerial Triangulation) is to measure the position and orientation of an airborne mapping sensor so that each pixel or range can be georeferenced to the specific map projection system without any kind of ground information collected in the field (Mian et al., 2015).UAV mapping is efficient to generate the newest map where does not allow to approach such as disaster area.The accuracy of on-board GNSS in UAV is however significantly lower than the accuracy of GPS used in general aerial surveying due to its own specification.Thus, related further researches are required to improve the accuracy of map.
This study aims to propose a convergence approach between data acquired from different types of sensors on a van-type investigation platform and UAVs of NDMI and to assess their applicability for timely natural and man-made disaster mapping and monitoring.

Investigation Vehicle for Rapid Disaster Mapping
The investigation vehicle onboard multi-sensors of NDMI have been developed for rapid mapping and response in emergency disaster situation.It is a specialized van mounting on a terrestrial LiDAR system, 5 low-resolution optic cameras with 2M pixels, a high resolution optic camera with 16M pixels, a thermal camera, a mobile weather observation device, and a pothole detector.

Functions of Investigation Vehicle onboard Multisensors for Disaster Management
Multi-sensors onboard investigation vehicle of NDMI has following variety of functions; -Multi-sources data acquisition and object identification without limitation at day-and night-time or regardless of weather conditions around disaster site -Rapid approach for disaster site and real-time communication with HQ -Timely sharing of various collected data in emergency -Real-time weather condition observation and analysis for disaster site such as wind speed, wind direction, temperature, humidity, atmospheric pressure, precipitation, visibility, etc.
-Environment analysis and contamination mapping for explosive incident site by atmospheric dispersion model -3-dimensional profile simulation and additional outputs generation acquired by 3D point clouds of terrestrial LiDAR -Drive-by detecting the potholes of road surface and analyzing vulnerability

UAV System
Unmanned Aerial Vehicles (UAV) with various types of onboard sensors is considered to be a cost-effective tool for largescaled aerial mapping.Direct geo-referencing of UAV sensors measuring the position and orientation of a built-in camera enable to create map products in real-time and rapidly so that it is suitable to utilize for disaster response applications.DJI Inspire1 is a common commercial quadcopter capable of capturing 4K video and high-resolution aerial photos.It is equipped with multiple sensors and autopilot to perform stable and autonomous flights.The built-in camera is mounted on an integrated gimbal to mix stability and weight efficiency.DJI Inspire1 on-board optical camera is Zenmuse X5.It is a mirrorless camera with the standard MFT interchangeable lens mount.It designed specifically for aerial imaging and also can capture ultra-clear 4K video at up to 30 frames per second as well as a still image with 16M (4,608×3,456) pixels.Its pixel resolution is about 3.7μms.In case of taking a picture in 150m height, GSD is calculated as about 4cm.The square target of 60cm × 60cm is expressed in the size of about 15 × 15 pixels on a photograph taken at 150 m height.

Study Area
The study site is selected as NDMI building and around its site which has various topographical features and easy to cooperate with experiment.Before capturing aerial photographs using UAV, 27 targets were installed at regular intervals around the NDMI building.In order to perform accuracy evaluation of direct geo-referencing and AT (Aerial Triangulation) by the bundle adjustment, GCPs (Ground Control Points) need to be installed around NDMI site.The control targets on the ground can be identified easily in aerial photographs.Generally, the shape of target is combined with circle and cross.In this study, target is made as the shape of square mixed circle and a cross.This process is necessary to improve the quality of final threedimensional models derived from UAV imagery.In particular, when using a non-metric digital camera for the purpose of surveying and mapping, it is very important to estimate accurate IOPs through the camera calibration experiment.In order to correct the distortion of the camera lens, calibration plate as shown was produced and taken a series of photographs in various directions.
Through the camera calibration experiment, the IOPs were corrected as shown in the table 3 and corrected imagery without lens distortion was obtained as below.

Image Acquisition
For evaluating the accuracy of geo-referencing for the captured from on-board Zenmuse X5 camera of DJI Inspire1 we planned to capturing aerial imagery at three different heights of 80m, 100m and 120m.A total of 142 images were acquired at an altitude of 80m, 69 images at an altitude of 100m and 69 images at an altitude of 100m, respectively.

Accuracy Assessment of Direct Geo-referencing
Direct Geo-referencing is a suitable process to create of accurate map products rapidly from UAV aerial imagery with minimal GCPs.This technology is efficient to produce the newest map rapidly where does not allow to approach such as disaster area needed emergent response.Direct geo-referencing in UAV photogrammetry is to measure the position and orientation of an on-board camera directly so that each pixel can be georeferenced to the Earth without GCPs.It requires precise location and attitude information for the on-board camera mounted on the gimbal of UAV.The accuracy of the GNSS/INS mounted on the UAV, however, is not enough to utilize the information as it provides.A complementary task is necessary for improving the accuracy.Direct geo-referencing by using the IOPs of camera calibration (upper Table 3) is performed without using the self-correcting algorithm provided from PhotoScan TM .
In this paper, the positioning accuracy is evaluated for two direct geo-referencing methods using collected UAV imagery at the heights of 80m, 100m, and 120m, respectively: 1) Direct geo-referencing by self-calibration, 2) direct geo-referencing with fixed accurate IOPs derived from indoor camera calibration test.The position accuracy of direct geo-referencing by self-calibration showed that a position error was about 29.5m at 80m, 45.2m at 100m, and 36.9m at 120m height.On the other hand, a positional error of the direct geo-referencing proposed in this paper was about 4m regardless of the flight altitude.The proposed direct geo-referencing with precalculated IOPs shows higher accuracy than existing direct georeferencing with self-calibration of the PhotoScan TM .

GNSS Field Surveying
After GCPs were installed around NDMI's site, the centric point of the targets on the ground was surveyed using RTK (Real-Time Kinematic) GPS.The absolute coordinates of the GCPs were obtained with a precision of 1cm approximately.
Points Longitude

Accuracy improvement using bundle adjustment
Though the GNSS/INS mounted on the UAV is mainly utilized for stable flight navigation, it can be reused to improve the positioning accuracy through post-processing.The image ray in photogrammetry connects an object space point, the perspective centre of an image, and the projection of the point on the image.The bundle adjustment based on the collinearity equation establishes the position and orientation of each bundle using the rays in each bundle and the given GCPs.
In this paper, we implemented geo-referencing with fixed IOPs and optimized EOPs through bundle adjustment and analysed the accuracy of the geo-referenced ortho-mosaics.The positioning error by geo-referencing using GCPs with PhotoScan TM estimated about 2.9m at 80m flight height, 2.5m at 100m height, and 2.4m at 120m height.On the other hand, a positioning error by bundle adjustment proposed in this paper was about 0.4m at 80m flight height, 2.5m at 100m height, and 2.4m at 120m height.

3D Model Reconstruction
Through the geo-referencing results, 3D point cloud data were obtained at the test site.Approximately 20 ~ 30 million points were generated according to the flight altitude.The generated point cloud data enables digital terrain model generation in raster formats such as three dimensional mesh or triangulated irregular networks (TIN).Depending on the flight altitude, a digital terrain model with 4 ~ 6 cm resolution can be produced.
Ortho-images below were generated using three dimensional terrain model derived from accurate geo-referencing results.It means that ortho-rectified image with about 20 ~ 30 cm resolution can be produced according to the flight height.

Matching Points Extraction
The matching points are needed to align the ground LiDAR point cloud with the UAV point cloud.As shown in Figure 4, we selected points that can be identified by the same point in two point clouds such as the corner of the building, the edge of the parking line and so on.The data processing of point cloud obtained Terrestrial LiDAR performs in an arbitrary coordinate system.In this paper, the 7parameter similarity transformation was applied to perform the ground LiDAR point and UAV coordinate transformation.The mismatching error in transformation with up to 1m caused a large gap between point cloud obtained from Terrestrial LiDAR and UAV.In order to reduce the error, the existing transformation equation is adjusted with height constraints like below.Assuming the scale was the same, it could be removed.Considering there was no rotation, the parallel movement was considered only in the Z direction. (1) (2) Where, is the average value of Z The spatial accuracy error by the existing method was about 98cm in the X-axis direction and about 23cm in the Z-direction.
On the other hand, our proposed method with the constraints of the height could be confirmed that the maximum error was greatly reduced to about 7cm.

CONCLUSIONS
This paper refers a convergence approach between data acquired from different types of sensors on the van-type investigation platform and the UAVs of NDMI and to assess their applicability for timely natural and man-made disaster mapping and monitoring.This shows some methodologies to improve geo-coding accuracy of multi-sources data collected from UAV and Terrestrial LiDAR including direct geo-referencing, traditional bundle adjustment, and data registration between the different kinds of data as a pre-research for assessing applicability of rapid disaster mapping.
In order to evaluate its applicability for rapid disaster mapping, we pre-tested for NDMI site in Ulsan, Korea.For the enhancement of the direct geo-referencing accuracy of UAV imagery captured from on-board camera of DJI and the creation of more accurate map products, camera IOPs refinement and bundle adjustment were also performed with minimal GCPs.
As above-mentioned UAV data registration with LiDAR point clould, integration technology of data collected from remote sensors on satellite, the unmanned-and the manned aircraft, and a variety of ground platforms is expected to improve accurate damage analysis and produce add-valued disaster maps rapidly for the purpose of disaster management.

Figure 1 .
Figure 1.Study area (NDMI site) 3.2 Camera Calibration mounted on UAV Camera lens calibration is a process to calculate the IOPs of the camera.The parameters of interior orientation are the location of principal point (x0, y0), focal length (f), lens distortion parameters (K1, K2, P1, P2).This process is necessary to improve the quality of final threedimensional models derived from UAV imagery.In particular, when using a non-metric digital camera for the purpose of surveying and mapping, it is very important to estimate accurate IOPs through the camera calibration experiment.In order to correct the distortion of the camera lens, calibration plate as shown was produced and taken a series of photographs in various directions.Through the camera calibration experiment, the IOPs were corrected as shown in the table 3 and corrected imagery without lens distortion was obtained as below.

Figure 3 .
Figure 3. 3D point cloud (left), DSM (centre), ortho-image (right) of test site 5. 3D POINT DATA INTEGRATION USING UAV IMAGERY AND TERRESTIAL LIDAR 5.1 Terrestrial LiDAR Point Cloud Acquisition NDMI operates an investigation vehicle mounted RieGL VZ-2000 to obtain high-precision three-dimensional LiDAR data at pre-or post-disaster dangerous areas.It can be used to extract terrain and monitor for steep collapse risk, old roads where the ground is vulnerable to sink, and atmospheric hazardous materials.VZ-2000 with on-board GPS/INS is installed on the vehicle and can directly acquire high-precision point clouds with absolute coordinates.We obtained point data around the building of NDMI.

Figure 4 .
Figure 4. Matching points for multi-source point clouds registration 5.3 Point Cloud Registration between UAV Data and Terrestrial LiDAR Data

Table 1 .
Onboard sensors of investigation vehicle, NDMI

Table 4 .
Coordinates of GCPs and CPs by GNSS survey

Table 5 .
Registration error occurred by transformation equations