DISASTER DAMAGE INVESTIGATION USING ART I FICIAL INTELLIGENCE AND DRONE MAPPING

: This study aims to testify the applicability of UAV photogrammetry and artificial intelligence (AI) for the management of natural disaster. Recently artificial intelligence is considered as an emerging tool for recognizing disaster events from aerial imagery of drones. In this paper, we present firstly the approach related to use of AI techniques for disaster detecting and identification. Secondly, we suggest small easy-to-use UAV-based investigation procedure for natural disaster damaged area in the phase of disaster recovery in Korea. Finally, we evaluate the mapping accuracy and work efficiency of drone mapping for disaster investigation application through comparing with traditional investigation work process which was dependent on labor-intensive field survey. The resolution ortho-image map of within less 5cm of GSD generated by aerial photos acquired from UAVs at the altitude of 100m~250m enabled us to check damage information such as facilities destroy or the trace of soil erosion around the river flooded and reservoir collapsed area. The photogrammetry-based drone mapping technology for the disaster damage investigation is expected to be an alternative approach to support or replace the labor-intensive disaster site survey that needs to investigate the disaster site quickly and timely.


INTRODUCTION
The strength and the frequency of natural disaster have been increased by recent severe global climate change and rapid urbanization over the world in recent years. In general, damage caused by natural disaster that are closely affected by topographical or meteorological factors can be reduced through taking preparative and preventive measures. For the effective and systematic disaster management, far-reaching researches and promising technical development have been carried out actively to minimize the damages by natural disaster using advanced observation platforms and observation sensors as follows: satellite-based and aircraft-based mapping platform, drone-based LiDAR, high-precision mobile mapping system (MMS) etc. (Mian et al, 2015, Oh et al, 2017, Rehak et al, 2013. UAVs, also called a drone mounted small and precise built-in sensors is an easy and cost-effective emerging observation platform for producing large-scale maps. Deploying a drone and collecting data after a disaster are widely and commonly, but the challenge is how to translate these data into actionable information. Another major driver of drone use is the application of Artificial Intelligence (AI) which is empowering drones to become smarter and more autonomous. With rapid development of computer vision and AI technology, drones are able to extract meaningful information from aerial imagery they capture leading to innovative functions such as object recognition and detection, classification, automated mapping and so forth. The AI algorithm applied on the imagery from drones rapidly processes data and was able to automatically detect, count and evaluate the state of affected facilities, flooded farmlands or missing person (Oren & Vertiy, 2020). Despite the dramatic growth of these technologies, socioeconomic losses and casualties caused by natural disaster have been increased in South Korea. The average cost of recovery by natural disaster is estimated about 400 million US dollars per a year over past decades. Most of the damage by natural disaster around the Korean Peninsula has been caused by flood and landslide such as a typhoon, a heavy rainfall for the rainy season. Occurring natural disaster in Korea, the local and the central government should investigate the damaged sites promptly, analyze quantitatively the extent of damage, and establish an appropriate disaster recovery plan in accordance with Framework Act on the Management of Disasters and Safety. Since 2013, Korea's National Disaster Management Research Institute (NDMI), responsible for implementing R&D related to national disaster and safety management, has studied for disaster scientific investigation (DSI). DSI, a highly organized framework to find the root cause of disasters, aims to implement, monitor and feed disaster profiling results through state-of-the-art forensic technologies back. As the feasible operational tools for DSI, NDMI has started to adopt and operate the various types of investigation platforms and devices: a MMS-type specialized vehicle, Unmanned Aerial Vehicles (UAVs), ultrasonography detector, rebar detector, etc. (Kim et al, 2018). Since flood events by Hurricane Katrina in 2005, a variety of drone have been used in the wide applications for disaster management. In Korea, it has been used by disaster response agencies for various type of disasters or incident events. Following is a drone deployment list used in response to latest natural disaster in chronological order of South Korea.
• Typhoon Prapiroon 2018: Both rotory and fixed wing drone were used to determine the flood and landslide damage in Boseong, South Jeolla Province. About 2,400 hectares of farmlands flooded in South Jeolla Province and North Chungcheong Province. Cumulative precipitation during this period was 236 mm ~ 327.5 mm over South Jeolla Province.
• Kyeongju Earthquake 2018: This was the first reported use of UAS on Earthquake disaster in Korea. DJI Inspire 2, a rotory wing drone, were used to investigate liquefaction phenomenon by Earthquake around Pohang in the southwestern of South Korea.
• Typhoon Mitag 2019: NDMI used drones to provide photogrammetric mapping of the flood damage by torrential rainfall around Uljin and Busan in the southwestern South Korea. Quoting KMA, Uljin, North Gyeongsang Province recorded rainfall rates of 104.5mm the highest since the authorities began compiling the data in 1971. At least 14 fatalities were occurred as result of Typhoon Mitag.
• Typhoon Jangmi 2020: DJI Inspire 2 and Mavic2 Pro used to investigate flood damage and mapping around Jaecheon in the centre of North Chungcheong Province and Geumsan in the South Chungcheong Province. At least 42 facilities occurred after 49 days of heavy rains in South Korea, with the country's longest monsoon on record in 2013 causing more flooding, landslides etc.
Disaster response can implement in a timely manner with AI algorithms. AI technology allows us to quickly analyse imagery and inform decision-making rather than spend time on the laborious task of identifying damaged areas.
The purpose of this study is to assess the applicability of UAV photogrammetry and artificial intelligence applying for the management of natural disaster. In this paper, we present firstly the approach related to use of artificial intelligence techniques for disaster detecting and identification. Secondly, we suggest small UAVs-based investigation procedure for natural disaster damaged area in the phase of disaster recovery in Korea.

Conceptual Approach for Disaster Damage Investigation
In order to identify the root cause of a disaster occurrence and investigate damage (Figure 1), the NDMI (National Disaster Management Research Institute) uses cutting-edge technologies and equipment such as drones and special vehicles to monitor the disaster accident site, analyse collected information using AI, spread the situation, and support decision-making (Kim et al, 2018(Kim et al, /2019.

Figure 1. Conceptual diagram of this study
In this paper, we suggest the two phase approach for damage information auto-detection and field investigation by natural disaster. At the initial stage, we introduce for artificial intelligence (AI) techniques for autonomous disaster detecting and identification. At the second stage, we implement small UAVs-based investigation procedure for disaster damaged area in the phase of disaster recovery in Korea ( Figure 2).

Auto-detecting of Disaster Info. using AI
AI techniques in the disaster damage investigation process can perform firstly an appropriate image recognizing and classifying task considering the characteristics of the disaster damage type. AI technology is also able to bring added values to the drone mapping process. When mapping disaster-damaged areas, image classification, object detection and image segmentation are the most important computing tasks. Disaster classification by AI is to classify an image or video into one or several sets of predetermined classes. In mapping, this classification is used to categorize geotagging-imagery. Disaster information detection is a task that enables the computer to find the damaged region within an image. These tasks are key for mapping as it analyses what is in a drone image, locates it, and plots it on a map. AI algorithms applied to computer vision tasks have improved data analytics on AI models trained to identify a specific object or region. the application of AI is therefore relevant when a large amount of data needs to be analysed. Machines can quickly analyse thousands of pictures, identify patterns and assemble the insights so that humans can do much more targeted decision making (Oren and Verity, 2020).

Disaster Damage Investigation using Drone Mapping
The increase of drone applications in civilian sectors is partly linked to advances in hardware, increasingly powerful batteries, increasing quality of images, and increasing capabilities of transporting. Therefore, drones are considered as an essential platform that can monitor specific areas in the prevention stage, communicate the situation of the site first in the event of a disaster, and precisely preserve traces of disaster in the recovery stage. In general, disaster management consists of following four steps: prevention, preparation, response, and recovery. When natural disaster occurs in Korea, disaster damage investigation led by local government is carried out as follows; disaster damage reporting, field investigation, damage analysis, recovery planning and implementation. According to the Disaster Recovery Guideline under Framework Act on the Management of Disasters and Safety, the heads of the local government in damaged area by natural disaster should promptly investigate the disaster site within the legal period (up to 28 days) and report damage level for facilities to the central government on National Disaster Management System (NDMS), and then establish a detailed recovery plan and implement it. If there is more damage than the recovery cost support criteria of the central government determined by the financial index of the local government, the central disaster safety headquarters will investigate directly and then judge whether or not to declare special disaster area. Local government should register the damage states of facilities by natural disaster on the disaster register of the NDMS, and all procedures in the disaster recovery phase for the disaster management such as the damage investigation of the disaster site, the damaged level analysis, the recovery cost calculation, and recovery implementation, etc. are managed through NDMS (Kim et al, 2019).

Specification of Drone Used
The small rotor wing and the fixed wing drones were adopted in the paper for the natural disaster site investigation. Drones for investigating damages occurred by natural disaster were selected focusing on aspects of safety, performance, and usability. DJI's Inspire2 is a commercial drone with a relatively stable flight performance and has a battery capacity of about 25 minutes with longer endurance flight than the Inspire1. Zenmuse X5S camera is mounted with a 3-axis gimbal which is suitable for taking aerial photography for drone mapping. A fixed-wing FireFly6 being capable of a vertical take-off and landing, is a rotor-tilling type drone on a flight mission, so it does not need a plenty of runway space for take-off and landing, which is advantageous for disaster field operation. A fixed wing drone enable users to carry out mapping mission with relatively stable and efficient flight performance, so it has advantages for long flight mapping in wide area disaster area. Sony A6000 optical camera mounted on FireFly6 TM has a built-in 24.3MP APS-C sensor that allows users to collect high-resolution aerial photographs with a 20mm lens and a GSD of 2.36cm at a flight altitude of 120m. Recently, it is possible to acquire relatively precise flight information of the drone through a postprocessing kinematic RTK (PPK) technology. In addition, a variety of multi-sensing data around disaster area can be collected from the multi-spectral sensor or the thermal camera depending on the purpose of applications (Kim et al, 2019).

Data Processing for Drone Mapping
Drone mapping is a simplified aerial photogrammetry-based data processing procedure using ultra-small built-in MEMS of a drone, though the accuracy of the IOPs or the EOPs for the nonmetric camera mounted on the drones is somewhat poor. Through the iterative bundle adjustment process using high capacity computing technology, drone mapping can generate the high-precise 3D terrain model and the ortho-image similar to the accuracy of the conventional aerial photogrammetry by optimization of IOPs and EOPs. In this study, the process of drone mapping is as follows: flight planning, GNSS-RTK surveying, aerial photos capturing, keypoints extraction and image matching, point cloud extraction, DSM/DEM generation, and finally ortho-image generation. Before generating DSM and DEM, accurate 3D point clouds were generated by extracting points of interest, optimizing camera models optimization, and improving the accuracy of image matching. In order to conduct drone mapping, we used Pix4Dcapture and FireFlyplanner for flight planning and aerial photographs acquisition, and Pix4Dmapper and Agisoft Metashape for aerial image processing acquired by a drone (Kim et al, 2019).

Natural Disaster Investigation and Mapping Using Small Drone
Aerial photographs of damaged areas were collected from onboard optic camera of drones and post-processed by drone mapping procedure to assess the quantitative damage level. We planned to capturing aerial imagery at flight heights of 100~150m. On the field survey of damage by typhoon Prapiroon, a total of 1110 aerial photos were taken at 5 damaged sites: 264 images at 2 landslide sites, 579 images at 2 flooded areas, 267 at a reservoir collapse site, respectively. At the time of Typhoon Jangmi attack, 182 images were acquired at a flooded area in Jaecheon city and 725 images at a flooded site in Geumsan county, respectively. The accuracy of the mapping using the small drones depends on various factors such as the terrain of the mapping area, the precision of the on-board sensors of drone, the image overlap/sidelap rate, weather conditions, flight speed and stability of drone, GNSS surveying conditions, etc. In the case of mapping using geo-tagged image information obtained in autonomous flight mode from barometer and GNSS /INS mounted on the drone without GCPs, the previous related literature says the accuracy of drone mapping using small drones is expected about 1~2 times of GSD horizontally and 1~3 times of GSD vertically for a correct estimation model (Pix4D, 2019). In addition, drone mapping using GCPs can improve the geo-referencing accuracy with cm-level, although there are some differences depending on the distribution and the number of GCPs and the accuracy of GNSS surveying. In this study, the horizontal accuracy of the result generated by performing drone mapping without GCPs in a dangerous and difficult-to-reach disaster sites. In the case of drone mapping of a mountainous region with an irregular altitude variance at the flight altitude of 200m, the maximum horizontal position error was about 9 m. In the flat area, the maximum error is measured with 1 ~ 2.3 m. The damage analysis by natural disaster is carried out using 3D map data (Point Cloud, DSM/DEM), Ortho-imagery based on location-based GIS data derived by using Pix4D Mapper and Agisoft Metashape (Kim et al, 2019).

Damage Analysis Using Drone Mapping
Many river facilities were lost during long heavy rainfall and floods and debris occurred at this time was run off into agricultural area. For investigate these areas, 249 aerial photos were totally taken and produced 3D DSM/DEM and an orthoimage map with a spatial resolution of 2 cm level. The drone mapping area was about 90,778.1 m 2 and debris run-off area into the agricultural land on the mapping area was analyzed to be about 4,390.6 m 2 . In addition, inundation and loss of facilities occurred mainly in agricultural area and around river facilities such as bridges, river weirs, bank protection facilities, access road along the bank of river, etc. The quantitative flood damage analysis results; flooded area in farmland, levee leakage, and collapse distance etc. can be utilized as objective investigation data for establishing natural disaster recovery plan.

Figure 7.
Flood damage analysis using drone mapping

Damage Analysis Using Drone Mapping
This study was suggested the approach using of artificial intelligence techniques for autonomous disaster detecting and classification as mentioned previously. The Cascade Mask R-CNN model was adopted by reviewing the recently published various segmentation models. Cascade Mask R-CNN is an advanced model of Cascade R-CNN, which expresses the area of the object to be detected as a bounding box, and extends to instance segmentation that detects individual regions of each object. For learning data, each class of flooding, collapse, soil erosion, and landslides were extracted for from still images and video sequences provided by the NDMI. VGG Image Annotator was used for learning data, and learning and verification were performed by dividing 8,528 data extracted from a total of 1,957 images including learning data of the project conducted in 2019 in the ratio of 8:2.

CONCLUSTIONS
The purpose of this paper is to assess the applicability of UAV photogrammetry and artificial intelligence (AI) for the management of natural disaster. As mentioned previously, AI is considered as an emerging tool for recognizing disaster events from aerial imagery of drones. In this paper, we present firstly the approach related to use of AI techniques for disaster detecting and identification. Secondly, we suggest small easyto-use UAV-based investigation procedure for natural disaster damaged area in the phase of disaster recovery in Korea. Before drone-based aerial surveying, the field survey can be performed with DGPS RTK for GCPs setting-up around disaster site. In this paper, we generate three dimensional terrain information and high-resolution ortho-imagery and then analyze quantitatively damage level by natural disaster using commercial UAVs and drone mapping technique. Finally, we evaluate the mapping accuracy and work efficiency of drone mapping for disaster investigation application through comparing with traditional investigation work process which was dependent on labor-intensive field survey. The resolution ortho-image map of within less 5cm of GSD generated by aerial photos acquired from UAVs at the altitude of 100m~250m enabled us to check damage information such as facilities destroy or the trace of soil erosion around the river flooded and reservoir collapsed area. The photogrammetry-based drone mapping technology for the disaster damage investigation is expected to be an alternative approach to support or replace the labor-intensive disaster site survey that needs to investigate the disaster site quickly and timely.