TOWARDS SEAMLESS INDOOR-OUTDOOR POSITIONING: THE IOPES PROJECT APPROACH

The management of emergencies require the use of multiple resources that must be coordinated to achieve the best possible results. For a good decision-making process, the availability of timely and reliable information about the variables on which such process rely is crucial. Among these variables, the ability to track the position in the field either outdoors or indoors of the members of the emergency teams it is of special importance. The IOPES project targets at improving an existing, already operational emergency management system where the tracking of operative staff is integrated. This paper concentrates only in the positioning aspect of IOPES which encompasses other subsystems, such as portable communications or fast mapping and describes the approach adopted by the project to perform such integration. This includes the concept itself, the hardware selected and well as the algorithms used to implement a portable, lightweight positioning device able to provide seamless indoor / outdoor positioning that will make possible the real-time tracking of personnel in the field. Promising preliminary results for mixed indoor-outdoor trajectories are as well presented. * Corresponding author


INTRODUCTION
Natural and man-made hazards have a tremendous impact from the environmental, personal and economical point view on the communities of people, affecting them globally and not distinguishing whether these are better protected or more vulnerable ones. According to 2020-IDMC-GRID report (IDMC-NRC, 2020), 2000 naturals disasters were documented in 2019, affecting 140 different countries. These hazards caused 24,9 million displacements of affected people and were responsible for a high number of casualties. Moreover, the number of natural disasters (no matter their type, e.g. hydrological, meteorological and climatologic or geophysical) is showing a positive trend in the last decades (Giordan, 2018;Guha-Sapir et al., 2017).
Natural hazards have destroyed and severely damaged a huge number of building during the last decades. Hereafter, a brief list of different natural events occurred only in Europe is introduced as well as their side or cascade effects to show their impact in terms of casualties, building or infrastructure damage. In Italy, the earthquake in the Umbria region (1997) damaged 8,000 buildings, while the one in L'Aquila (2009) damaged 11,000. Earthquakes in 2016 in Central Italy destroyed the city of Amatrice, while an avalanche produced by an earthquake in Rigopiano (2017) killed 29 people. In Greece, the Earthquake in Cephalonia Island (2014) damaged 1,525 buildings; 444 buildings (2.8% of building stock) were damaged in Lefkada Island (2015), and 1,115 in Lesvos Island (2017). 2 people died in Lefkada and 1 person in Lesvos. In Portugal, on 1998, in Faial Island (Azores), an earthquake killed 8 people, 1,700 people lost their houses and many buildings and infrastructures were damaged. During October 1997, near 1,000 landslides occurred in San Miguel Island (Azores, Portugal), killing 29 people. Several houses and bridges were partially or totally destroyed. In Spain: a subsidence located in Sallent (Catalonia), produced the collapse of several buildings and the majority of the remaining ones were severely affected, requiring the complete demolition of the neighbourhood. In 2013, severe floods in Vall d'Aran (Catalonia, Spain) caused damage in several buildings and in the road and communications infrastructures. In Iceland: the earthquake in South Iceland (2008) damaged 2000 buildings, roads and bridges were deeply affected. In 2010, the eruption of the Eyjafjallajokull volcano resulted in direct ash fall estimated at around 250 million tons. The ashfall was persistent for about six weeks, with the rural areas in south Iceland most severely affected. Hundreds of inhabitants had to be evacuated from the disaster area three times during this period, due to e.g. risk of flash floods.
However, man-made hazards have also a huge impact, possibly requiring a high number of resources from the emergency services to cope with them. For instance, in 2015 and 2016 the number of interventions of Catalan firefighters due to urban fires were around 12,700 and 13,000 respectively in the region of Catalonia (Spain). In Denmark, fireworks accident in Seest was one of the largest Danish accidents in recent times. It hit the southwestern Kolding suburb Seest on November 3, 2004. A gigantic explosion cost a voluntary firefighter's life, sent about 85 people to the hospital and forced the evacuation of more than 2,000 people from over 750 households.
The management of an emergency triggered by natural or manmade disasters, requires the use of multiple resources, both human and material, that must be coordinated to achieve optimal -or at least, the best possible -results. A good, efficient coordination process (managing) helps to deliver best responses, better suited to alleviate or solve the problem being handled. Coordinating means deciding, and decisions should always be taken relying in as much information as possible. The information on which such decisions are based must reflect the situation's picture, being handled as accurately as possible, leaving the less possible room to uncertainties-that is information must be reliable. This implies also that information must be collected timely and put to the disposition of the decision-making personnel as soon as possible; outdated data serve for no purpose or, even worse, may induce to erroneous actions.
Taking decisions based on timely, reliable information implies having a mechanism to collect it, distribute (transmit) it, store it and present it in an understandable manner to those involved in the coordination, decision-making process. This means deploying a system or a set of systems able to perform tasks as surveying, mapping and monitoring, involving sensors, communication systems, data processing and visualization equipment, as well as specialized algorithms. In particular, near real time or real time evaluation and control of the emergency event, if possible, is very important during the emergency or response phase. This monitoring is a key to perform a better risk assessment scenario (Giordan, 2018). Geomatics technologies such as positioning and satellite, aerial or RPAS mapping, together with the advance on Information and Communication technologies, provide new insights for the monitoring of these hazards and for improving the decision-making process.
This work is developed in the frame of the IOPES project, cofounded by the European commission -Directorate-General Humanitarian Aid and Civil Protection. This project addresses the major natural and man-made hazards affecting the EU reported also in the "Overview of natural and man-made disaster risks in the EU" (2017): flooding, earthquakes, volcanic activities (eruption), landslides and subsidence, severe weather conditions (storms, droughts) and industrial accidents or damages to individual buildings due to explosions or fire, by combining a set of innovative Geomatics and ICT tools.
The goal of the concept presented in this paper is to increase the safety of civil protection and emergency teams working in disaster and post-disaster (either natural or man-made) scenarios, as earthquakes, flooding or fires. The members of these teams are constantly exposed to situations that may put their lives at (even death) risk. Such risk is significantly increased due to the lack of knowledge of the environment they work in -typically, confined spaces as damaged buildings. Increasing such knowledge should have a direct impact on their security. Here, "knowledge" refers to information about the places (either outdoors or indoors) where these teams work.
In this context, this paper presents a concept and the preliminary results for a low-cost, lightweight positioning system, for civil protection and emergency teams working in disaster and postdisaster emergency scenarios. Such a concept envisages continuous, seamless tracking in both indoor and outdoor environments; this would be possible because of the low geometric requirements set by emergency teams: knowing what the floor and room where they are located is enough.
An initial state-of-the-art and feasibility study was partially presented already by the authors in (Angelats and Navarro 2017). However, technological advances and miniaturization of GNSS positioning systems together with the introduction of Commercial-of-the-self (COTS) visual-inertial odometry devices providing object positions with high rates and very few latency (VIO) since 2017, confirms the soundness of the approach and deserve further investigation. This paper contributes to it by presenting a methodology to fuse GNSS and VIO derived positions to have a seamless indoor-outdoor trajectory.
To do so the paper has been organized as follow. Firstly, related research on Mapping and Indoor positioning as well as further details of the IOPES project are presented in Section 2. Then, in Section 3, the proposed methodology to provide seamless outdoor and indoor positioning, detailing Hardware, Software and operational aspects, is presented. The preliminary experimental results are presented in Section 4. Finally, Section 5 summarizes the conclusions of the proposed approach and discusses future improvements 2. BACKGROUND

Related research
There exist, indeed, solutions to first map an area and then track people in booth indoor and outdoor environments. In the case of outdoors, there are nowadays many companies operating Remotely Piloted Aircraft Systems (RPAS) producing high quality cartography very quickly. Note, for example, that even the Copernicus Emergency Management Service (CEMS) offers a fast mapping service for emergencies using either Sentinel imagery or RPAS with a response time of about 48 hours - (CEMS, 2015;Ajmar, 2015); locating someone outdoors is also routinely performed: common solutions rely on Global Navigation Satellite Systems (GNSS) receivers or an hybridization of these with Inertial Measuring Unit (IMU) sensors. There also exist solutions for positioning in confined places, but these rely on pre-deployed infrastructures (as WiFi emitters or cameras, among others) that will not be available when working in post-disaster scenarios since these may be located anywhere. Solutions for indoor mapping based usually on Light Detection and Ranging (LiDAR) sensors carried either by humans or even by terrestrial robots (Kruijff-Korbayová, 2016), but these are not suitable to operate in post-disaster spaces; walls may have collapsed, debris or holes may impede their normal operation and, in the case of human carriers, operating in such situations is, doubtlessly, dangerous.
In spite of being a tool widely used lately by most of the applications needing precise and robust positioning, the wellknown GNSS has some drawbacks, such as the need of good environment conditions, as clear lines of sight; confined spaces or deep canyons (either natural or urban) are the typical environments where GNSS receivers are not the best technology for achieving precise positioning results. As already mentioned, this limitation is usually overcome (Mautz and Tilch, 2011;Dardari, 2015) by means of the ad-hoc deployment of different kinds of emitters (as Wi-Fi, ultra-wideband or even visual beacons) that are complemented with the corresponding receiver, which, aided by the appropriate algorithms is able to estimate a solution. The emitters play the role of landmarksalso known as anchor nodes -since these have been deployed in known positions. This is not, however, the only approach used indoors. (Leutenegger, 2013;Veth, 2011) describe the use of combined IMU data and other sensors (as monocular, stereo or RGB-D cameras).
Visual inertial odometry (VIO) is a technique to derive trajectories combining visual and inertial measurements. It is a field of research actively studied in recent years with significant advances in terms of computational and accuracy performance (Scaramuzza, 2019;Huang, 2019;Tschopp, 2020).

IOPES project
The IOPES (Indoor-Outdoor Positioning for Emergency Staff) is a two-year project co-funded by the European Commission involving 7 partners from 5 different European countries. IOPES targets at strengthening the preparedness of emergency personnel by making them more responsive to disasters. This is so because IOPES seeks to improve an existing, already operational tool specifically dedicated to the handling of emergency situations. Furthermore, the ability to collect timetagged positioning information that may be therefore related to specific, significant events, facilitates the post-event analysis of information in order to derive new strategies / procedures or improvements to these.
The project is funded by the Union Civil Protection Mechanism (UCPM) whose goal is to "improve the quality of EU response capacities" as stated in its Annual Work Programme (2019). Besides that, the IOPES is also fully aligned with priority 4 of the Sendai framework for disaster risk reduction 2015-2030, "Enhancing disaster preparedness for effective response". The project involves the combination of several technologies ( Figure  1) including RPAS-based fast mapping, emergency management, portable communications and positioning technologies. IOPES aims to provide continuous, timetagged information about the location of Civil Protection Emergency Teams (CPET), either indoors or outdoors. This paper does not cover all the technologies involved in IOPES but only those related to the reliable indoor / outdoor positioning of emergency staff".

Figure 1. IOPES project technologies
As stated in section 2.1, it is true that solutions for indoor positioning in general do exist, but their dependency on some infrastructures make these unsuitable for emergency and disaster management. On the other side, it is the authors' belief that using the current technologies and algorithms it is possible to develop a system able to solve precisely this problem for the specific case of civil protection and emergency teams. The concept presented here targets at a solution to overcome the difficulties stated above, a low-cost, lightweight, unobtrusive, portable device to be carried afterwards by civil protection and emergency members when inspecting an area affected by a disaster.
One factor making this approach possible is the low geometric (accuracy and precision) requirements set by the regular operation of civil protection and emergency personnel; basically, their members want to be sure about the floor and room they are in (based on authors' personal communication with end users). This would translate to accuracies around 1-2 metres and precision between 30-50 cm. The other factor is the recent advances in visual-inertial odometry/SLAM technology, such as Intel T265 tracking device that allow to trajectories based on visual and inertial data in real time on system on chip modules.

PROPOSED APPROACH
The positioning sensors used to build such system are a tracking camera and a GNSS receiver. The camera tracking would combine data coming from an IMU and stereo fisheye cameras to provide, by means of visual-inertial odometry, the location, either indoor or outdoor, of emergency staff. The addition of a GNSS receiver immediately enables such system also for outdoor environments and for providing global positioning coordinates. A data fusion algorithm that relates GNSS and camera-based positioning is the heart of the system. This algorithm allows to keep the positional drift of the tracking under control when good GNSS conditions are available. The output frequency of the fused data is 10 Hz. All these components (both hardware and software) are mounted and running on a light, battery powered system-on-chip computer (also with low power requirements). The next subsections introduce further details from the HW, SW and operational point of view.

Hardware
The system devised by the authors is a portable, low weight positioning device made of COTS hardware components, able to be mounted on a helmet or embedded in the clothing. A battery powered System-On-Chip (SOC) board will be used to run the sensor fusion & positioning algorithm and to integrate the required sensors -the tracking camera and the integrated GNSS module. This last module could be the ublox NEO-M9N (ublox, 2020). It has been chosen because it is able to provide meter-level accuracy and receive signals from up to four different single-band GNSS constellations. This GNSS chipset have been selected with the idea to have reliable GNSS position coordinates even in weak GNSS conditions such as urban canyons, exploiting its capability to select the best signals. The output rate of positions is 20 Hz, matching the targeted operational fused positioning mode rate (10 Hz).
Regarding COTS cameras, it is worth to note that a high evolution of this technology has taken place during the last years, especially concerning RGB-D cameras and RGB stereo vision cameras. Examples of the first type of cameras are (1) the Intel Realsense (Keselman et al., 2017) and (2) Microsoft Kinect 2 (Lachat et al., 2015). The second option has been discarded because the computation in real time of the Visual-Inertial-Odometry trajectory is delegated to the SOC module, thus increasing the required computational burden. Recently, a new family of cameras have appeared in the market, such as the Intel T265 and T261 ones (Intel, 2019) and the Microsoft Hololens 1 st and 2 nd generation (Microsoft, 2019), which perform a trajectory computation in a local reference frame using on-board SOC modules and, providing the user the locations using a predefined API or SDK tool. The Intel T265 was selected for this work due to a much lower price than Microsoft's Hololens.
The Intel T265 tracking camera provides the current pose (position and orientation) with an output rate of 200Hz. The The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B4-2020XXIV ISPRS Congress (2020 pose is estimated using Visual Inertial Odometry proprietary algorithm, running on board, combining images from two fisheye lenses and measurements from an IMU Sensor (Tsykunov, 2020).
Finally, an unobtrusive SOC (also with low power requirements) would be desirable to complete (and integrate) the set of components making the system. Its task, to provide the necessary computing resources and storage capacity. The word unobtrusive above means lightweight and small footprint in this context, since the positioning device must not be a nuisance to its wearers. Obviously, the low consumption requirements lead to longer operational times, thus reducing the need to replace batteries so often. From the computing power standpoint, a powerful GPU is not need, since the visual-inertial odometry computations are performed by the camera device itself. A candidate for the SOC is the Raspberry Pi 4 (Raspberry, 2020).

Sensor fusion SW approach
The key part of the approach presented in this paper is an algorithm able to fuse GNSS and VIO positions providing a single trajectory, regardless of whether it runs indoors, outdoors, or both. The flowchart of the proposed algorithm is detailed in Figure 2.
Firstly, a common temporal reference frame is necessary to deal with data coming from these two sources -the internal clock of the SOC will be enough for the purposes of this concept. Secondly, the VIO camera is considered in our approach as the primary sensor, due to the capability to provide a trajectory in both indoor and outdoor areas. The GNSS solution is used to convert the positions provided by the VIO from local to global coordinates and to correct the temporal drift of the VIO data.

Figure 2. Flowchart for the sensor fusion approach
If the portable positioning device starts to acquire data in an indoor area, the system is designed to store the VIO positions (in local coordinates) and three attitude angles in the internal disk till absolute positioning is available. When the positioning device moves to an outdoor area and the number of GNSS satellites allow to provide a set of reliable GNSS based positions, a local to global transformation can be estimated, and so VIO positions in global coordinates can be provided since this moment. The criterion to consider a GNSS position as valid is that the Precision Dilution of Precision (PDOP) of the GNSS solution has a value lower than 4, thus indicating a good GNSS satellite geometry.
To estimate the local to global transformation, at least two valid consecutive GNSS positions in different locations are needed to have not only the transformed positions themselves but also to estimate the heading angle. In our approach pitch and roll angles are assumed to be close to zero although this may introduce some positioning errors. Then, a rotation matrix can be computed and then a global VIO position, the lever arm offset between the GNSS and VIO devices and the local VIO coordinates can be estimated using this rotation matrix.
If the rotation matrix between GNSS and VIO devices is known and consequently global VIO positions are available, the GNSS based solution is used (for outdoor areas) to estimate the temporal drift of the VIO. This is done comparing the coordinates of the global GNSS position and the global VIO estimates for a temporal window of n seconds. After these n seconds, a linear drift is estimated for each positioning component.
The positional drift is then applied to the newer global VIO positions till a new positional drift is computed. If no valid GNSS positions are available for the new window (transition from outdoor to indoor), the older positional drift is maintained until new GNSS positions become available (transition from indoor to outdoor). The n number is a parameter that must be set up prior to the use of the system.
Finally, the stored local VIO positions are also converted to global coordinates to be able to have the historical track of global positions. This track can be used during the management of the emergency or once it is over to perform a post-mortem analysis.

Operational aspects
From the operational standpoint, using the system in indoor and outdoor environments implies no difference for the members of the emergency teams. Their position will be computed in realtime using the approach described in section 3.2. Ideally, this position should be sent to the control team outside the building or affected area so they can track the position of the personnel. This, obviously, implies the use of communications that, as stated before, will not be described in this paper. Note that, depending on the conditions indoors (dust, no lightning, smoke, etc.) the portable device might be unable to compute positions.
Again, when indoors, the possibility to have 3D models and 2D floor maps together with the location of team members in the hands are invaluable tools for the control team: these may be used to guide the teams inside buildings using classic communication channels -for example, telling them where to find holes, collapsed walls, debris, or any other obstacles that might interfere with their task. These 3D models or 2D floor maps can be previously generated using Structure-from-Motion photogrammetric tools and imagery acquired with RPAS suited to operate in confined spaces such as Flyability's (Flyability, 2017), which is an example of RPAS used successfully in projects requiring flying indoors. Alternatively, several commercial solutions that exploit the potential laser-based SLAM are available (Nocerino 2017, Calantropio 2018.

Dataset description
A campaign was carried out to validate the individual performance of the positioning sensors (visual-inertial odometry and GNSS sensors) in an area comprising buildings as well as areas with poor GNSS conditions. The campaign was done at the Parc Mediterrani de la Tecnologia (Castelldefels, Spain), a technological campus that include several buildings, areas with good GNSS coverage and other with strong multipath conditions, allowing to evaluate the transitions between indoor and outdoor areas.
Two different tests were performed (Figure 3), to evaluate the Visual Inertial Odometry solution provided by the T265 tracking camera and to the assess the potential of an already available low-cost, low consuming GNSS receiver embedded into a Quectel BD96 board. The sensors in both tests were carried out by hand, trying to simulate close to real environments conditions. The VIO test (Figure 3) started from a point with a known position and coordinates, and so being able to generate a global VIO trajectory. For the VIO test, a person started walking from a known point inside a building, then it moved to an outdoor area. Next, it entered another building with a long corridor followed by a walking period in an outdoor area. This path was closed by returning to the first building, finishing at the initial position. The GNSS test (Figure 3) started in an area with good GNSS visibility, then the person moved to another one with strong GNSS multipath Then the person entered an area that finished in the parking lot with very good GNSS conditions. After that the person walked back to enter a large building with no GNSS signal. The last part of test was done in an area with good GNSS visibility to close the walking loop.
Both tests were done around noon with outdoor sunny lightning conditions. The indoor areas where illuminated by ambient lights. However, lightning conditions between indoor and outdoor areas were significantly different, and thus this might introduce some additional errors in the estimated VIO trajectory. The required time to perform the tests was 7.6 minutes for the VIO test and 7.4 minutes for the GNSS test.
To evaluate the planimetric performance of the solution provided by the sensors, 8 reference points with known ground coordinates ( Figure 3) were used. Five points were used for evaluating the VIO solution, while four were used for evaluating the GNSS performance. The Root Mean Square Error (RMSE) for the planimetric coordinates was estimated to evaluate the absolute positioning performance. The height component's absolute accuracy was not assessed. However, for the VIO solution, it was possible to estimate the relative height precision for two different points using their estimated coordinates with a temporal shift of 6.3 and 7.18 minutes respectively.

Preliminary results
A graphic depiction of the positioning results from the VIO device and those from a single GNSS receiver can be found in Figure 4 and Figure 5 respectively. A preliminary analysis of the performance shows that the VIO solution can deliver outdoor/indoor positioning with a planimetric RMSE of 3.0 m and 3.3 m ( Figure 4) for a walk of about eight minutes. The mean of the relative error of the elevation is 1 m. Such value has been computed using two points only. Therefore, its significance is not high.  The results show a planimetric RMSE of 0.75 m and 0.68 m. Concerning the results, two different aspects need to be highlighted in relation to the evaluated GNSS receiver. The first one is the notable degradation on the quality of the GNSS solution in areas with strong multipath (path section between two buildings, Figure 5). The second one is that the GNSS receiver provides some indoor positions (outliers) after going out when leaving a long indoor corridor. These results suggested to the authors that another GNSS with multiconstellation capability is needed. For this reason, the use of U-BLOX M9N receiver (introduced in section 3.1) together with a pre-filtering strategy (section 3.2) to check the quality of GNSS data is proposed. Further validation with the proposed GNSS receiver will take place in a near future.

CONCLUSION AND FURTHER RESEARCH
In this paper we have presented the approach developed in the frame of the IOPES project to provide seamless indoor and outdoor, continuous positioning or tracking of emergency staff. The approach takes advantage of the recent advances in VIO COTS devices and proposes a fusion mechanism merging the VIO (local) and GNSS (global) positions. The paper introduces the SW approach to perform the fusion of these data To do it, both sensors and algorithms are presented, and positioning sensors are preliminary assessed. The preliminary results show the potential of using the solution of the VIO device as the primary positioning source. GNSS based positioning solutions are used for providing global coordinates and for modelling the temporal drift of the VIO with GNSS filtered positions. This GNSS error-based modelling seeks to keep the positional drift under control.
However, drawbacks derived from the specific work conditions usually present in the target scenarios such as sub-optimal lightning, or the presence of dust, usually found indoors in many kinds of natural or man-made disasters, have been not evaluated. Finally, it is worth to note that these results are, although promising, only preliminary. Further research will serve to evaluate more deeply the validity of this approach. It is foreseen to test the fused solution using reference data and assess the impact of sub-optimal lightning and different walking dynamics.