WILDFIRE DETECTION AND DISASTER MONITORING SYSTEM USING UAS AND SENSOR FUSION TECHNOLOGIES

Based on a research program founded by the German Ministry of Traffic and Data Infrastructure, the use of 5G mobile network for modern advanced technologies should be developed and tested. This paper shows the integration of UAS, Multiple Sensor Integration, autonomous UAS Missions and online Geodata handling for the detection of wildfires in a rural region of Saxony / East Germany and the strategy for making use of the upcoming 5G Network. In our project we go a step deeper and combine VIS and Thermal IR cameras in a co-registered way to overlay the images, transfer data of multi-observation poles into a center and perform an automated analysis to detect smoke or hotspots and add to a first estimated localization. The technological advance applied on real project areas means an important step for further applications of UAS and Sensor Fusion Technologies.


INTRODUCTION
For the introduction of the 5G mobile network, the German Ministry for Traffic and Data Infrastructure, Mr. Andreas Scheuer has founded several research projects to initiate strategies for making use of the upcoming 5G Network with its very fast mobile data transfer. Based on this research program, the use of 5G mobile network for modern advanced technologies should be developed and tested. We have setup a strategy for wild fire monitoring using fixed observation points and autonomous UAS to finally localize the hot spots. The project area is a in a rural region of Saxony /East Germany close to the border to Poland. Wildfires are a serious issue in this flat and dry area of the project region. In the last decade, droughts have become quite common; an effect discussed as a first result of global warming. More than 50% of the project area is covered with pine and oak trees on sandy soils; a very vulnerable ecosystem for wildfires. A few towers enable human observers to monitor for smoke in the region during such dry periods. The research area also covers a military training area for tanks, where an own firefighter station is located to fight fires caused by shooting practice. A subset of the area is an open mine for coal, where fires are observed by automatic cameras for smoke detection. Both areas are critically endangered by wild fires especially in the coal mining areas; whereas in the past a fire lasted one week on a coal seam. This project has just started and is the first step of a feasibility study with first and single test beds. In 2021, we aim to perform the next step parallel to the upgrading of the mobile network 5G with a full implementation of sensor and UAS. There are several international projects on the way and our project aims to integrate the real time data management into a real time action plan. To collaborate with the various researchers is a must.

WILDFIRE DETECTION
This kind of monitoring also takes place after thunderstorms or other critical events. In this flat landscape the localization of fires and their smoke column is difficult. The determination of the direction is semi-accurate, while the estimation of distance is problematic. This also results in problematic guidance of the firefighting teams. The observation from different points may improve the localization, but the coordination is not well managed and a central management center is lacking. Besides that, a great critical issue remains navigating fire control with reference to location factors: different villages only having voluntary fire departments, the private coal mining company LEAG AG in Nochten, and the military training area of German Armed Forces with a size of 175 km².

Actual Situation
Monitoring of Wildfires are still managed in a very traditional way and limited to critical weather periods. Today, the observation is managed by human observers on elevated points or observation poles by monitoring for smoke columns. In wider areas, so called Fire-flights are undertaken to search for smoke columns using small airplanes e.g. Cessna 172 and other small or UL aircrafts. Very often private hobby pilots provide these type of jobs and report findings to fire stations. That is in most cases based on private activities and rarely integrated in a systematic monitoring system. The use of satellite images is used only for monitoring bigger fire disaster but seldom for the finding of upcoming wildfires. The need of innovative technology is on the way and some fire-fighting stations make use of UAVs to get information after an alarm from elevated points, however, the systematic integration is something that must be done. Very often the alarming system is individually, manually and cost-intensive and not automated.

Static Wildfire monitoring
To improve the basic observation in context with 5G installation, the mobile transmission towers will be used as part of the observation pole network. Each pole is equipped with a high resolution RGB cameras for automatic detection of smoke columns and for night view with a Thermal Camera to detect the hotspots. The RGB cameras are at least a 20 MPIX cameras with a zoom lens and light sensitive CMOS chip. The refresh rate should be at least 1 FPS. This camera will be mounted in a 2-axis gimbal to capture a panoramic image sequences of up to 360° to monitor the entire landscape for smoke columns. The second axis is needed to tilt the camera for close and remote views especially while zooming for the findings. It will also be used to manually control the gimbal for operator interactions. The system will have 2 modes, a fully automated mode for scanning the landscape and a manual mode to have access by the operator in the center. The images will be transferred via 5G to the control center and the data will automatically be analyzed for changes especially considering smoke columns. The use of adapted algorithms and machine learning will be part of the software that analyses findings. Besides that, the operator has a real-time view of the incoming data and can influence the gimbal to zoom in for manual observations. The gimbal has precise incremental sensors and the entire setup is calibrated. This way, the orientation of the camera in heading and tilt can be measured with 0.05-degree accuracy. The positions of the towers are precisely known and the estimated distance between them is about 10 km. This way, an estimation of the distance should be around 500m for the remote areas, even better for closer ones. By using several observation points, the data from cross sections should give a final localization of better than 200 m. But this needs a real-time combination of the orientation of the cameras and the intersection of the measurements. The online analytics will detect the smoke, use the orientation of the cameras and intersect the position of the finding on a map. For this purpose, a precise geodatabase exists with updated information of the infrastructure. This way a first localization in real-time is possible. More difficult is the monitoring in the night time. A thermal camera is connected to the RGM Camera on the same gimbal. However, the resolution is typically very poor (640*512 pix). The localization is far more difficult and it is not sure if the hotspots inside the forest are visible from the poles. But if there are anomalies detected, the combination of the different observations may help for the location of fires.

Firebird
The big innovation is the use of an autonomous operating UAS that carries RGB and thermal cameras. The type of the system should be a vertical-take-off-landing (VTOL) due to the lack of a runway needed for the fixed wing UAV and the demand to hover over strategic places to observe. The entire area should be equipped with four to five of these Firebirds. The Firebirds are ideally placed at the fire stations to enable battery refresh and other maintenance. One station shall be part of the open mining company, the other inside the military training camp and operated by the German forces.
We developed a long range UAS that is capable to fly for one hour and carries both cameras in a three-axis gimbal. The UAS is an electric driven quadcopter. Due to the demand of a longer flight time than the actual available units with 20 to 30 minutes and the higher payload capacity, we optimized the construction for its performance.
The best efficiency is given by a quad rotor concept with large rotors and low RPM of the motors. Higher operational voltage results in less diameter of the cables, which also saves weight. Figure 4: UAV Therefore, we use 28-inch rotors with motors of 120 KV. The rotors are positioned as a pusher, which means they look downward and are located under the arms of the quadrotor. There is a noticeable aerodynamic advantage which also results in better efficiency and flight time.
As a power source the typically used Lithium Polymer packs are replaced by cylindrical so-called Lithium Ionic cells, which are now more attractive. The LiPo sample pack in the picture above (left side) has a capacity of 1550mA, the LiIo pack on the right offers 3000mAh So at nearly the same size and weight, the LiPo offers 22,94Wh, the LiIo 43,2Wh The UAV is constructed in such a way that the power units take higher voltage and lower current.
These cells offer some advantages, whereas the most interesting for our project can be summarized as:


Safer handling as there is no tendency to deformation and "puff"  Higher energy density  Larger lifespan with 500 -1000 charge discharge cycles  Larger voltage range than LiPo`s  Can be stored fully charged  Meanwhile available for high current applications (a single 18650 cell can offer up to 35 amps)  Significantly lower cost than lithium-polymer  Best self-discharge rate, which is typically around 5% in the first 4 hours after being charged, then falls to around one to two percent per month!  No active maintenance required Using RTK technology and precise incremental sensors in the gimbal, the localization is within an accuracy of a few meters. The UAS tries to find the real location while flying to the before observed finding.

DATA MANAGEMENT -REAL TIME GIS AND AI
The data management in the Firebird Project needs certain requirements. The real time GIS is based on a central Geodatabase, which is a system of basic 3D data captures and analyzed beforehand combined with the newly captured data from the UAVs. The basic 3D data consists of various information about the project area, which means land cover and infrastructural data. Furthermore, a topographic road network (meaning to contain information about road junctions and accessibility of them) has to be taken into consideration, as a detected fire needs to be reached by the responsible fire fighter units in the shortest time. Therefore, the completion of all roads, the accessibility with fire trucks, the length and driving time of each road segment etc. have to be gathered. Important objects like the fire fighter stations and gas stations have to be loaded into the system. Appropriate rescue plans, considering the wind field in the area, points of interest and the possible evacuation of people in the zone can be enhanced through the usage of real time GIS information.
The secondary requirements of the data management are based on Machine Learning Techniques, which are implied in the phase of fire detection. The training dataset has to be created for setting the computer algorithm. Several Machine Learning Techniques can be implied in this process to differentiate between a real fire and another possible incidents. This can be done by implying values on specific parameters, which might reveal an anomaly of temperature or the minimum size of a detected object that is recognized as a potential fire. The dataset used for training must be given in a certain size to assure a sufficient number of available samples to represent the real life's complexity (Allen et al. 2016.) Figure 7: Scheme of a wildfire monitoring and defense system (König 2020.) The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2020, 2020 XXIV ISPRS Congress (2020 edition)

MANAGEMENT OF THE FIRE FIGHTING
The firefighting management depends on the real time GIS system and has the urge of up-to-date information. The information management has to be fast to ensure that the teams heading to the forest fire know the size of it and how to combat the flames, if they need further support or even the wind direction to take preventive measures. When firefighters are on-site, the UAV is responsible for a real time monitoring of the fire to assure that the chosen technique is appropriate and to decide on further processes. The control of the wind direction can give important information about people being exposed to smoke, These people should be alerted to close their windows for a certain time and get informed when the air quality improves again. Furthermore, it is necessary to guarantee human safety on-site, so the possibility of people entering the danger zone should be seen by the drone.
The procedure is as follows: 1. Thermal camera detects anomaly in the covered area. Possible forest fires are detected and this information is given to the UAV 2. UAV leaves firefighter station and heads to the detected area 3. Parameters are detected, pictures and videos are sent in reallife to the firefighter station (5G needed) 4. Decision if firefighters take action In case immediate action is required: firefighters head to affected area (with use of the rescues plans and the information given by the UAV) and extinguish the fire in an initial phase In case no action is required: UAV returns to starting position, ready for the next discovery mission and waiting for thermal information The fire fighters can make use of additional linked devices as GPS and Cameras to stream the data of the trucks while approaching the area. This can help to get additional warnings and actions started on the fly. It is also important to connect the various Firefight stations in order to have the fastest possible access and may be from different start and routing points to optimize the defense of the wild fires.

OTHER APPLICATIONS
The presented technology has great potential for other areas like flood monitoring. Riverine floods that regularly affect Germany could be detected and information about affected people and property could be given. Especially along the bigger rivers and the dams parallel to them should be monitored.
In mountainous areas secondary disaster as mudflows, hang slides and slide dammed lakes can increase the risk dramatically. Especially the monitoring of wild river beds with a high sediment load may increase during heavy rain falls to mud flows that are several times more dangerous than a water flood is. Especially the alpine region with is affected by such mass movements during wet periods. Since hundreds of years cities and villages are faced with that risk. It would be a great opportunity to use the firebirds for the exact opposite case of disaster, as the firefighters still need to reach critical points to evacuate people or reach properties for protection by barriers against the water. Even if this requires different parameters and training datasets, the operational character remains similar. Further, optical imagery of the project area could be used to assess damages after a storm. Germany is a storm prone country and experiences severe storms with wind speeds over 140km/h. The high quality image could be used for a damage assessment and blocked road segments or paths could be detected easily in a short period of time.

CONCLUSION
Wildfires are a serious and increasing issue in Europe. Modern communication infrastructure (5G mobile network) can assist in a wireless data handling for rapid response to the teams. A proper setup of geodata capturing, central data server with AI improve the quality and success of early defense of wildfires before a disaster appears. The combination of different data sources is one of the basic tasks to fulfill, to get an autonomous mission flown by the UAS is more a legal and administrative issue than a technical one. All these facts have to be taken into account before the system is stepping from the test-phases going to be operational use. There are several important benefits that will be created with our project including an improved forest management, reduced risk and damage of forest and the including ecosystem services and enhanced security of the local community.