REAL-TIME AERIAL IMAGERY FOR CRISIS MANAGEMENT: LESSONS LEARNED FROM AN EUROPEAN CIVIL PROTECTION EXERCISE

Regular international civil protection exercises are an important part of the European Civil Protection Mechanism. One such exercise, called IRONORE2019, took place in September 2019 in Eisenerz, Austria, with the aim of training international cooperation of relief teams in case of an earthquake. In parallel to this exercise, the European project DRIVER+ conduced a Trail in order to test novel solutions for civil protection. The German Aerospace Center (DLR) provided aerial imagery as well as derived map products to the project and the exercise, which were also made available to the Bavarian Red Cross, among others, as exercise participants. In this way, products developed using the latest scientific methods could be used and tested in practice. The valuable experiences from this operational use, which are explained in this article, serve the enhancement of the processes and products and will be implemented in the future in order to further support disaster management.


INTRODUCTION
Over the last years the occurrence of climate change-related events such as heavy rain falls, floods, thunderstorms or wildfires continuously increased in frequency and severity and therefore poses an enormous challenge for the crisis management sector. In order to overcome this challenge and to strengthen the resilience during crisis, the development of relevant and accessible innovations becomes crucial. In 2014 the European Commission founded the project DRIVER+ (Driving Innovation in Crisis Management for European Resilience) in order to cope with current and future challenges caused by the increasingly severe consequences of natural disasters and terrorist threats. The main aim of the project is the development and uptake of innovative solutions that are addressing the operational needs of practitioners dealing with crisis management (DRIVER+, 2020). To achieve this goal, four Trials were organised and carried out, based on the gaps in crisis management and the needs of practitioners.
The Trial Austria covered a simulated earthquake scenario around the city of Eisenerz and the Erzberg mine. During this Trial, crucial phases such as the situation and damage assessment and subsequent disasters triggered by the earthquake, e.g. a chemical spill, where simulated and played through in a crisis center and in field exercises. Organized by the Austrian Red Cross (ARC) and the Austrian Institute of Technology (AIT), the Trial was conducted as a multi-day field exercise and hold in parallel to the large scale European Civil Protection exercise IRONORE2019 in Eisenerz, Austria, from the 12th to the 14th of September 2019.
IRONORE2019 was co-funded by the European Union Civil Protection and had the aim to test the preparedness of response * Corresponding author to an earthquake disaster in the framework of the European Civil Protection Mechanism (IRONORE2019, 2019). The exercise therefore focused mainly on revising and, if necessary, improving the response time of local emergency services, the request for assistance and opening up the European Civil Protection Mechanism to receive international assistance and the integration of incoming assistance into the Austrian response system. During the exercise teams and experts from Austria, Hungary, Slovenia, Germany, Greece, Poland, Slovenia, Spain, Sweden and Romania practiced for a real disaster situation at around 30 different simulated field exercises.
The German Aerospace Center (DLR) was a partner of the DRIVER+ project and provided for the Trial Austria an innovative solution for the support of situational awareness during crisis. A main component of this solution was the acquisition of images via an airborne camera system and the provision of the processed data in near-real time through a direct data transmission. During the Trial, the generated orthorectified image mosaics were visualized in the crisis center. Reference maps generated by DLR's Center for Satellite Based Crisis Information (ZKI) (ZKI, 2020) were updated with the latest aerial imagery from our camera system and provided as an additional information source. Since DLR is also a cooperation partner of the Bavarian Red Cross (BRK), which was a participant in the IRONORE2019 exercise, the aerial photographs and map products were used for their work in the field. The following article gives an insight into the results of the Driver+ Project and from the cooperation with the BRK, provides lessons learned and outlines the necessary future developments derived from it.

REAL-TIME AERIAL IMAGE PRODUCTS FOR CRISIS MANAGEMENT
In the following, the airborne monitoring system used and the specific adaptations made for the Driver+ project and the flight campaign in Austria are described. We then present the products that can be derived from the captured aerial images and show what kind of information we can automatically derive from the data.

Real-time Airborne Monitoring System
The real-time airborne monitoring system considered in this paper was developed by DLR and consists of an optical camera system, an onboard processing unit, a microwave data downlink and an ground processing unit. The 3K camera system consists of three non-metric Canon cameras (EOS 1Ds Mark III), which are equipped with 100 mm ZEISS lenses. Two of the cameras are looking sideways so that they just overlap, whereas the nadir looking camera was deactivated in this campaign (see Figure 2). The field of view (FOV) of the camera system is in this configuration a 37 • along track and 13 • across track. The flight height was planned at 2000 m above ground, which leads to a ground sampling distance (GSD) of 13.0 cm. For the flight campaign in Austria, the 3K system was mounted on the universal research aircraft Dornier Do 228-101. The exercise site located in Eisenerz, Austria has an area of about 200 km 2 , which was mapped with several overflights between 07.09.2019 and 13.09.2019. Most points of interest defined in IRONORE2019 and DRIVER+ were acquired with 3K aerial images in two modes, the 3D acquisition and real-time mapping mode. The difference between these modes are the image acquisition rate (1 Hz vs. 0.25 Hz ) and the distance between flight strips (60% vs. 10%) to increase on the one hand the overlap between the images for stable 3D generation and to lower on the other hand the data rate for downlink by lowering the overlap.
On board the aircraft, four computers control the image acquisition, the orthorectification, the data storage and the transmission to the ground. The data transmission is realized via a microwave downlink with a data rate of around 10 Mbit/s and a range of around 100 km to the ground station. An illustration of the ground antennas can be found in Figure 3. More technical details about the data downlink can be found in (Kurz et al., 2011). The mobile ground station consists of the above mentioned microwave ground antenna and a data processing and validation infrastructure. The orthorectified images received on the ground can then be passed on directly to the rescue forces, e.g. via a web-based portal, in near-real time and can processed further on request in order to derive additional information and to provide further valuable products such as a digital surface model (DSM) (more details in the following sections).

DSM Generation
The 3D acquisition flight was used to generate a digital surface model during the campaign. Due to the increased acquisition frequency, downlinking of the imagery during the 3D flight was not possible, and processed off-line after the airplane has landed. After improving the image orientation with bundle block adjustment, an orthophoto-mosaic and a digital surface model of Eisenerz and Erzberg were generated. A fast version of Semi-Global Matching (d'Angelo, Kurz, 2019) was used to produce height data by matching each image with the two previous acquisitions. After merging and elimination of areas where only few inconsistent matches were found, such as on water or partly visible areas near the image boundaries, a high quality digital surface model with a ground resolution of 30 cm was obtained within two days of the flight. The generated DSMs and derived products are shown in Figures 4, 5 and 8.

Map Generation and Reference Data
Up-to-date, high resolution imagery is crucial for emergency response and serves as an important information source for The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2020, 2020 XXIV ISPRS Congress (2020 edition)  decision-making. However, first responders who are unfamiliar with this type of data may feel challenged with interpreting the situation and damages from pure aerial imagery only or require additional information. This is where the ZKI of DLR comes in: To assist the response teams in this task, ZKI fuses satellite or aerial imagery with ancillary geo-data to give context to the images. In close consultation with the user, the areaof-interest (AOI) and required content are determined to suite the responders intended use. As result, information products such as maps (printed or digital) or web services are created.
Depending on the user's needs and priorities, ZKI further supplies professional image interpretation and analysis to highlight affected infrastructure or categorize damages (Lechner, Gähler, 2017). In addition to its mapping services, the ZKI team offers consultancy, technology transfer as well as user trainings.
As part of the DRIVER+ and IRONORE2019 exercise, two sequential mapping activities were carried out: At first instance authoritative data was acquired to rapidly compile reference maps of the deployment area. An overview map (1:30,000) of the entire region of interest was provided together with two local overviews (1:8,000) that focused on specific parts and three detail maps (1:1,500) showing certain AOIs on building level. All reference maps have been tailored to the user's demands and generated on the basis of geo-data from the province of Styria, supplemented by additional information acquired through OpenStreetMap (OpenStreetMap contributors, 2017) where needed. For the overview maps latest available aerial imagery, terrain information, rail and road transportation infrastructure (including bridges and gas stations), surface water network, administrative boarders and protected areas were shown, while the detail maps further included individual buildings and additional points-of-interest. Apart from the first-mentioned, all listed 2D maps could be extended by a corresponding 3D version due to fortunate availability of federal 3D geo-data (highresolution digital terrain model and building height information).
The reference maps were prepared and delivered as print (2D) and digital versions (2D/3D) already prior to arrival of the emergency responders on site. To facilitate better orientation and usability, the digital 3D product could directly be accessed through the interface of its 2D equivalent. Reference maps can be provided already about eight hours after receiving imageand ancillary geoinformation. Thus, quick access to adequate, up-to-date geo-data is a decisive factor in such rapid mapping activities. Open data initiatives by national and local authorities as well as standardization activities play a major role in supporting these efforts with high quality and easy to access data.
Following the flight campaign by DLR, the prior prepared reference maps could be updated with latest aerial imagery at little expense. The special acquisition plan further allowed DLR to process a very high resolution DSM of the Eisenerz/Erzberg region besides orthorectified images. This enabled the team of ZKI to update not only the above described 2D but also the 3D products with the latest information within short timeframes. Effort may vary subject to number of products, content and analysis needs, but took about two working hours in total for the example of this exercise. Considering the 3D maps, differences of reference data and the data update by DLR (consisting of DSM and orthorectified imagery) are reflected in the object-oriented approach on the one hand and surface-oriented approach on the other, both having advantages and disadvantages for use in map applications. This means that, in this case, reference and updated 3D scenes are different in nature and not necessarily designed for change detection, but rather meant to support emergency responders in different stages of deployment.
All digital maps were provided as interactive PDFs, as they have low usage requirements and thus advantages for portability and use across all user profiles. With providing even lightweight GIS (Geographic Information System) functionality, the GeoPDF format is an established solution within the ZKI services for sharing 2D map information and is well accepted by the user. However, the 3D PDF comes with some limitations in terms of data size and visualization capabilities, but serves as a good compromise for rapid information transfer. Modern GIS allow synchronized 2D and 3D views side by side that combine the advantages in presenting information of both environments. To overcome the aforementioned limitations especially for presenting 3D content, processed data might as well be distributed as web maps for mobile use or even entire GIS packages for specialists in the situation centers on request. This is however associated with higher requirements on internet connection on site, or software requirements and geo-data expertise respectively.

Automatic Information Extraction from Aerial Images
In addition to the acquisition of image data and the provision of maps, an important step is the automatic extraction of information to further support the rescue forces during the image interpretation phase.
The following examples should show what kind of information can already be automatically extracted from aerial images in post-processing. As this information can't yet be provided as finished products to the end users, it was not provided during the Driver+ trial or the IRONORE2019 exercise.
During IRONORE2019, an artificial landslide was created for demonstration purposes. Several vehicles were partially buried in the landslide. The artificial landslide was carried out in the iron ore mining area on a slope between two mining terraces. Stone material was slid down from the edge of one mining terrace to the next lower terrace between 10.09.2019 and 13.09.2019. Parallel to the chronological sequence of the artificial landslide, aerial photos were taken with the 3K sensor system on 10.09, 11.09. and 13.09., in which the individual events on the landslide were very clearly visible (see figure 4). Starting from the initial situation on 10.09.2019 without landslide, the organizers moved existing vehicles closer to the lower edge of the slope until 11.09.2019. During this period, the filling material was provided at the upper edge of the slope for the landslide and a tent standing nearby collapsed (visible in figure 4 a) to d)). In the following period until 13.09.2019 the provided filling material was then slid down the slope, so that the vehicles and the tent were partially buried (visible in figure 4 c) to h)).
The volume of the moving mass in the landslide can be derived from the differences dij in the terrain models from 10.09.2019 and 13.09.2019. First, the systematic offset (avg(dij) = −0.1m) and noise (σ(dij) = 0.44m) must be eliminated from the differences. Therefore, only positive and negative differences bigger than 0.72m (95% confidence level) are taken into account for volume calculation. From the positive volume of 720m 3 , which contains stone material and vehicles, the volume of the vehicles (volume of around 200m 3 ) must be subtracted. Finally, the moving volume of the artificial landslide results in 520m 3 .
Besides the detection of changes or the derivation of the moving mass volume, another important question during a landslide scenario is whether people are buried or could be buried under the ground. One indication, which could help answering this question, is the search and the identification of objects such as cars on site. Therefore we applied an algorithm for the automatic localization and detection of cars (Azimi et al., 2018). This algorithm is based on a novel feature pyramid network with multi-size convolution kernels to extract multi-scale strong and weak semantic features. This method can be applied on any image data regardless the scales or orientation of the vehicles or the image itself and is able to detected cars within seconds once the network is trained. Figure 6 shows the results of the automatic car detection for the landslide scenario on the 11.9.2019 and 13.9.2019.

USAGE OF AERIAL IMAGE PRODUCTS DURING IRONORE2019 AND DRIVER+ TRIAL IN AUSTRIA
In the following, an overview is given of the provided aerial image products and their exact usage in the context of the Driver+ Trial and the IRONORE2019 exercise, respectively.

Aerial Image Products for the Driver+ Trial
The practical part of the Driver+ Trial in Austria was carried out over two days. During these days, several simulated subscenarios were played through in a crisis center and partly carried out in field exercise. The sub-scenarios were predefined by the IRONORE2019 exercise. During each sub-scenario, practitioners from the Austrian Red Cross were confronted with a particular initial situation and should try to use all the provided solutions to get additional information about the current situation and to handle the simulated crisis as good as possible. In addition to the practitioners, several observers accompanied the different sub-scenarios. The observers were research and innovation professionals from the field of disaster and crisis management and evaluated the provided solutions based on their observations, but without any interactions with the solutions providers or the practitioners.
During the sub-scenarios, the practitioners could request aerial images from specific areas and where provide with single orthorectified images in near-real time and with mosaics covering the large scenes after a certain processing time. The visualisation of the mosaics in 2D and 3D was provided on a large screen. Next to the up-to-date image data, the practitioners had access to the 2D and 3D map products (described in Section 2.3) and could deploy them as reference data, mark certain areas or plan operations.

BRK Operations in the Context of IRONORE2019 and their Usage of Map Products
The role of the BRK during IRONORE2019 was to assist and support the ongoing rescue operations with rescue teams (modules). The first unit, a quick reaction team responding to chemical, biological, radiological or nuclear (CBRN) events, provided support with the decontamination of injured persons and supply of technical equipment for hazard containment and neutralisation. The second unit, called "SEG -G.I.L.T.", is a special team for heavy ground, infrastructure, logistic and transport which provides the assessment with drones and amphibian vehicles and works closely with the ZKI. Both units were involved in different scenarios and operated in tunnels, industrial facility and alpine environment. More precisely, they participated and supported during a simulated bus accident, a rail tanker crash, a chemical spill accident, a landslide and an mass casualty incident in a traffic tunnel. Thereby, the BRK trained the cross border support in case of distastes, e.g. in how to prepare the teams for the task of evaluating and updating the protocols and how to use the host nation support from the requesting country. Before the exercise, the BRK was equipped with printed and digital versions of the generated map products. The printed maps where especially important for the field work as commonly the technical infrastructure is poor during the first two days of a disaster and were used for the daily mission planning, briefing of incoming teams at the reception and departure center (RDC) and, by the commanders and for the exercise control. The digital map products found application in the operation room (Integrated command center) in Schweinfurt/Germany as support for the units on their way to and exercise and onsite. During the exercises, the field operators were constantly in touch with the command center who could cross check the position an provide further information about the deployment site. Figure 7 shows example of the usage of our map products during the IRONORE2019 exercise.

DRIVER+ Assessment of the Applicability of Aerial Image Data for Crisis Management
After the DRIVER+ Trial in Austria, observers and practitioners gave feedback on the aerial photography products. They all appreciate the products as a valuable source of information for situation assessment and planning activities and were impressed by the quality and resolution of our products. By performing several flights and by using reference data, a comparison between the situation on the ground before and after the disaster (here artificial landslides) was possible, and provided valuable information for the rescue teams.
The practitioners assessed the identification of the collapsed building walls from the nadir view images provided by us during the trial as a very difficult task. Therefore, they proposed the expansion of our products by images from various platforms such as Unmanned Aerial Vehicle (UAVs) or helicopters to provide oblique images for critical areas for a better assessment and search of damaged buildings. The idea would be to detect critical areas in the wide-view aerial images and then send UAVs or a helicopter to acquire oblique images on the critical areas only. Furthermore, due to the weather dependency of our products, the availability of data cannot be guaranteed and can therefore only be used as an optional support for crisis management.
Even though aerial images can be provided in real-time, practitioners and observers pointed out the problem of a correct and accurate data interpretation as an important challenge for the future. As rescue forces are commonly not trained on the interpretation of aerial or satellite images, trained specialists on site or automated analysis in real-time through artificial intelligence methods would be an additional support.

BRK Assessment of the Applicability of Aerial Image Data and Products for Crisis Management
The handling of the 2D and 3D products was easy for the end users, and after a short training period all participants could work with them. The 3D products were found to be particularly helpful, as such information can highly increase the safeness of rescue teams and enable the support of field operations from a remote operation room. More precisely, the resulting view of a foreign terrain made it easier for the control center in the country of origin to plan the deployment of resources and equipment, as well as the route to the deployment site. An example is illustrated in figure 8. It was also easier for the rescue teams on site and hence safer to orientate themselves in unknown terrain. The tools for measuring distances, heights and objects supported them in assessing the situation. Zoom function and detailed views were appreciated and found to be very suitable for the intended use. The printed maps were also used on site to brief the emergency services and discuss the situation.
On the other hand, the users of the digital products identified the need for acceleration of loading and updating the products. Especially devices with low computing power needed a lot of time to update the maps. Furthermore, despite the relatively simple handling of the map products, the advice of an aerial image expert on site in acute phases of a crisis could be helpful for further support in a operational planning. Also the automatic detection of crisis related points of interest would be a beneficial for the future. For this, lists of coordinates visualized in a map would be a desirable interface.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2020, 2020 XXIV ISPRS Congress (2020 edition)

CONCLUSION & FUTURE WORK
This paper investigated the usability and usefulness of real-time aerial image and map products for crisis management. Two independent user groups were provided with the image products during a simulated multi-day disaster situation in Austria organized and executed by the EU projects Driver+ and IRONORE2019. All users described the provided data as a useful and helpful support for crisis management, especially for the real-time situation assessment, the planning of operations on site or from a remote operation room as well as for increasing the safeness of rescue teams during field operations. Nevertheless, the evaluation also revealed the necessity for further developments such as the provision of automatic extracted information like damaged buildings or roads, detection of changes, cars and people. Furthermore the support by aerial image experts on site and the additional image acquisition from other airborne platforms such as UAVs for oblique views were identified as potential supplementation.