REAL-TIME UAV BASED GEOSPATIAL VIDEO INTEGRATED INTO THE FIRE BRIGADES CRISIS MANAGEMENT GIS SYSTEM

During a fire incident live airborne video offers the fire brigade an additional means of information. Essential for the effective usage of the daylight and infra red video data from the UAS is that the information is fully integrated into the crisis management system of the fire brigade. This is a GIS based system in which all relevant geospatial information is brought together and automatically distributed to all levels of the organisation. In the context of the Dutch Fire-Fly project a geospatial video server was integrated with a UAS and the fire brigades crisis management system, so that real-time geospatial airborne video and derived products can be made available at all levels during a fire incident. The most important elements of the system are the Delftdynamics Robot Helicopter, the Video Multiplexing System, the Keystone geospatial video server/editor and the Eagle and CCS-M crisis management systems. In discussion with the Security Region North East Gelderland user requirements and a concept of operation were defined, demonstrated and evaluated. This article describes the technical and operational approach and results. * Corresponding author.


INTRODUCTION
During a fire incident in an environmental area or complex industrial site it is often difficult for the fire brigade to build up a good overview of the incident A flexible view from the air provides a better overview of the fire itself, the surroundings and the presence and activities of people.It also can provide details on hot spots or leakage in direct support of the fire fighting activities.Small unmanned systems have the advantage that they can be used flexibly with a short reaction time and at dangerous or inaccessible locations.
As unmanned aerial vehicle systems are becoming more mature and easy to operate, these might be useful to fill in this information requirement.Many small UAS systems are on the market and can be equipped with a video camera.Many of them however are not operational systems in the sense that operation of the platform and sensor requires too much skills of the operator, and that the sensor information reaches not further than the control station of the operator.The information cannot be distributed and integrated into the users GIS based information system.
For the involved fire workers and coordination agencies it is important however that the system will be easy to use and that the information is well integrated in the crisis management system, so that it can be combined with all other information and distributed to the different users.
In a project Fire-Fly in the framework of the Public Innovation Agenda of the Dutch Ministry of Economic Affairs a project has been carried out with the Dutch Security Region North East Gelderland with the objective to demonstrate the usage of real time UAS sensor information in the command chain.See also Figure -1.Attention has been paid to the operational and information requirements of the fire brigade and to the geospatial integration of the video information into the GIS based crisis management system.The project ended with a demonstration in which the system was used and evaluated in three different scenarios.

SYSTEM DESCRIPTION
The system as developed for support of the fire brigades activities is described in this chapter.The user requirements are discussed and the derived system architecture, followed by a description of the different system components.

Fire brigade information requirements
Based on discussions with people from the fire brigade and a demonstration of technical capabilities at the start of the project a concept of operation and requirements to the UAS system could be specified.This was worked out in a system architecture as shown in Figure -2.The system is based on a UAS with visible and infra-red video cameras.Also attention has been paid to chemical measurements, but this is not part of this article.In this architecture the central point for the UAS coordination and information handling is the local command centre that is placed at the incident location.In this shelter a video server is defined that processes, archives and distributes the live video and products to client systems.Multiple UAS systems can be supported.One of the client systems is the Video Editor Station that is operated by an information analyst, who inspects the live video, generates products, and is a liaison for information requests to the UAS ground control stations.In the shelter also the plotter station of the crisis management system is present with which the video products can be retrieved from the video server and combined with all other relevant information.Based on the crisis management system local coordination of all activities can take place.All information however is also distributed to the other regional coordination levels and operational persons in the field.A number of relevant information products have been distinguished: the live video stream, video clips, geo-referenced or geo-corrected snapshots, video mosaics.The information manager also does have a capability to send request for the imaging of specific areas of interest to the ground control station of the UAS operator.
Interesting is that comparable UAS usage and requirements occur in the defence area.Also here real-time geospatial video is required to determine and analyse locations and objects of interest for intelligence, and to combine the sensor information with other information sources to support command and control.Different data standards have been developed in this context of which some also could be applied for this application.An example where the same ground station technology was used for this type of usage is the project Purple Nectar (Bransen, 2009).

Video processor
The raw video as received in the local command centre in a first step is formatted.In this step also some enhancement of geometric quality of the meta information is done.For the geospatial handling of the video it was decided to apply the military NATO standard STANAG-4609 format (NATO, 2009;MISB, 2008) in which meta information on the geospatial parameters of the platform and sensor are stored in the MPEG-2 Transport Stream.A Video Multiplexing System was developed with which flexibly several formats of video and meta data can be read-in and converted.

UAS systems
In this project video data from different type of UAVs was used.The system development was based on the usage of the Delft Dynamics Robot Helicopter (Jong, 2011).For the final demonstration also the Aerovironment PUMA fixed wing platform was coupled successfully to the system.Besides the NLR Pelican system from Ascending Technology were used, but finally not coupled to the information chain.

Video server
For the handling and distribution of the geospatial video and the generation of products the Keystone Image Management System is used as a basis.This is a server-based system that archives, catalogues, processes and delivers digital images and video from satellites and airborne digital sensors.The system is developed by Spacemetric (Edgardh, 2011).In cooperation with NLR adaptations and extensions are made for clients and research.Keystone is Java/Eclipse based and has a client-server approach in which the OGC standards are applied.Focus of the Keystone system is the accurate geospatial handling of the imagery.For video the standardized video and meta data is ingested, enhanced and stored in the server database and catalogue.With the Keystone Client the live video stream can be viewed or catalogue queries on the raw video or derived imagery products can be made to the server, after which the stored video or products can be viewed.

Video editor
As a basis for the Video Editor station the Keystone client has been used.See also Figure -3.Display of the live video is done in the client, both in a raw video window and a map window in which the video is displayed perceptively together with the aircraft position and in combination with other raster maps and imagery and vector overlays.A mosaicing function has been added in which a dynamic mosaic of the video is built up for a better overview and also static mosaic products can be generated and stored in the database.Figure 3. Keystone based Video Editor with database part, project overview and display of video, perspective map and mosaic Based on the video, products can be made, stored and distributed: mpeg clips and projected frames or mosaics as geotiffs.Distribution is done by an ftp server.Special attention was given to situations where the video is partly pointed over the horizon.In this case relative mosaics can be generated, based on relative image stitching.For these the geolocation information is stored in the exif fields.

CoT function
For commands to the UAS ground station controllers a Cursor on Target protocol is used (JEITA, 2010) that makes it possible to directly do requests from the crisis management system clients to the ground control station.A server guiding the requests was added to the Video server station.

Crisis management system
In the Fire-Fly project two crisis management systems are involved, Eagle (Neuvel, 2009) from Geodan based on ArcGIS and CCS-M developed by Nieuwland (Nieuwland, 2011).Both systems were extended with a separate video viewer with a coupled vector footprint on top of the map.Also the information analist is able to retrieve clips, images and mosaics products form the video server and distribute these to all users of the crisis management system.Figure 4. CCS-M crisis management system with video integrated imagery and footprint

CONCLUSIONS
A final demonstration and evaluation of the Fire-Fly system was held in May at the G4S training site in Delden.Two scenarios were exercised, one of a car accident with multiple cars inclusing a truck with chemicals, another with a fire in an industrial complex next to a railway.During the demonstration live geospatial video and GIS products based on it were made available in the local command centre, while the video products were distributed to the regional coordination centres and local fieldworkers by the crisis management system.The relevance of the data was judged positive by the involved participants.
In an earlier demonstration also scenarios for detection of leaking liquids, a smoke plume and missed people walking in forested area were executed.In all cases live geospatial video helped in the detection, localisation and support of actions.
Interoperability was demonstrated by the use of two different UAS and two different crisis management systems.It appeared that the helicopter and fixed wing systems both have there strong and weak points for fire brigade applications, related to endurance, range, static observation possibility.
Further enhancement of the video information (like stabilisation, contrast, superresolution, higher geometric accuracy) will contribute to the easier interpretation and use.
During the demonstration the UAS and video edit system were operated by technicians and not the fire workers itself.Further attention is required to ease of operation, robustness and operator training.Special point of attention is the In a next step further operationalisation of the concept needs to be worked out.The idea is to make one or more pilot systems available to the fire workers in order to gain operational experience based on which the system and procedures can be optimised.After this phase finally an operational system can be integrated in the organisation.

Figure 1 .
Figure 1.Flexible eye in the sky to support fire brigade Figure 2. System architecture