PERFORMANCE ANALYSIS OF THE IOPES SEAMLESS INDOOR-OUTDOOR POSITIONING APPROACH

Tracking the members of civil protection or emergency teams is still an open issue. Although outdoors tracking is routinely performed using well-seasoned techniques such as GNSS, this same problem must be still solved for indoors situations. There exist several approaches for indoor positioning, but these are not appropriate for tracking emergency staff in real-time: some of these approaches rely on existing infrastructures; others have not been tested in light devices in real-time; none offers a combined solution. The IOPES project seeks to solve or at least alleviate this problem by building a portable, unobtrusive, lightweight device combining GNSS for outdoor positioning and visual-inertial odometry / SLAM for the indoors case. This work, the third of the IOPES series, presents the analysis of the performance results obtained after developing and testing the first IOPES prototype. To do it, the operational aspects of the prototype, the real-life scenarios where the tests took place and the actual results thus obtained are described.


INTRODUCTION
Deciding is a very important component of the management of emergencies: events happen unexpectedly and must be faced promptly; resources, either material or human, must be distributed -hopefully in an efficient manner -throughout the emergency scenario. Deciding means reasoning when, how and what to do, and this reasoning must be backed by information describing the situation in the field. Since situations change constantly, information must follow this very same path, being updated in such a way that the global picture seen by the emergency managers is the right one, at least for as long as possible. Moreover, such information must be as accurate as possible too, to avoid wrong decisions and thus inappropriate responses. In short, information must be both timely and reliable.
Information must be therefore somehow collected, transmitted, and appropriately presented to the emergency managers in order to make their decision process not only possible but also effective. Several systems must be thus distributed throughout the field to perform the aforesaid tasks, which translate to activities such as the deployment of sensors and communication systems, the mapping of the emergency scenario, the processing of data using various types of algorithms to make them easily understandable by humans when they are presented on the appropriate visualization equipment. These systems comprehend Geomatics technologies such as positioning and mapping, either aerial or satellite-based, and advanced information and communication technologies.
As stated above, the collection, transmission and transformation of the relevant information must happen as fast as the situation changes; it should happen, in fact, in real-time or in near realtime to improve the monitoring, which is crucial (Giordan, 2018) to perform a better risk assessment of the emergency.
Continuous tracking of civil protection and emergency teams working in disaster and post-disaster emergency scenarios is still an open issue. While GNSS is a stablished solution for outdoors tracking, there is still no suitable solution for the indoors equivalent. Team managers know the building where their teams are working but not in which part of the building they are. The authors are working on developing a full system able to improve this situation. This paper presents the first results of a low-cost, lightweight positioning system developed in the frame of a project, co-financed by the European commission -Directorate-General Humanitarian Aid and Civil Protection.
Outdoors positioning is routinely performed nowadays by means of Global Navigation Satellite Systems (GNSS) receivers optionally hybridized with Inertial Measuring Unit (IMU) sensors. Despite its reliability, GNSS has some drawbacks such as the need of good environmental conditions. When these are not met (as, for instance, in deep canyons) GNSS is no more the best technology for precise positioning. Indoor navigation solutions also exist, usually relying on ad-hoc, pre-deployed infrastructures such as Wi-Fi, ultra-wideband or even visual beacons (Mautz and Tilch, 2011;Dardari, 2015), which will not be available (at least, everywhere) in post-disaster scenarios. Alternatively, techniques that combines visual and inertial measurements also exits (Scaramuzza, 2019). (Ramezani et al, 2017) suggest and proposes a visual-inertial odometry approach to improve conventional approaches by using visual measurements derived from omnidirectional cameras and multistate constraint Kalman filter based methods.
Solutions already exist for both outdoor and indoor positioning, but these are not appropriate to fulfil the requirements set for the IOPES (Indoor-Outdoor Positioning for Emergency Staff) project. Firstly, some of the indoor technologies rely on complementary infrastructures deployed in advance in the area where the positioning must take place. In the context of IOPES, indoor-outdoor seamless positioning must be available no matter what the situation is, that is, it must be self-sufficient, not depending on pre-existing resources. Secondly, other indoors techniques, based on visual-inertial odometry / SLAM (Ramezani et al, 2017), have not been tested in portable devices and real-time, like the one that this project intends to build. Last, but not least, no combined solution, providing seamless indoor-outdoor positioning, exist for such kind of light devices.
The IOPES concept seeks to fill this gap, building a lightweight, low-cost device for the emergency and civil protection teams working in areas affected by a disaster, during or after the emergency. Said device, already presented in (Angelats and Navarro, 2017) and (Angelats et al, 2020), combines the wellseasoned GNSS for outdoor positioning and the advances already available in visual-inertial odometry / SLAM materialized in devices such as the T265 tracking device to provide positions indoors. The result is a solution providing seamless indoors-outdoor positioning, suitable for civil protection and emergency teams.
At this point it is worth to remark that the requirements that the IOPES device must match are not as strict as in other situations requiring positioning. This is due to the type of application it is targeted at. Members of civil protection and emergency teams stated, during the phase of requirements collection, that being aware of the room and floor where the members of the team were enough for their purposes. This translates to an accuracy of about 1-2 meters and precision in the range 30-50 cm.
The concept was first introduced in (Angelats and Navarro, 2017). In (Angelats et al, 2020) the proposed methodology to provide seamless outdoor and indoor positioning, detailing hardware, software, and operational aspects, was presented together with the preliminary experimental results. This paper presents the results of the performance analysis as well as the conclusions for the first operating prototype performance in real scenarios.
To do so the paper has been organized as follows. Firstly, a short introduction on the IOPES project and technologies, together with the hardware, software, and operational aspects of the system, are presented. Then, in section 3, a description of the different real scenarios used to collect the data to estimate the performance of the system together with the experimental results, are presented. Finally, section 4 summarizes the conclusions of the proposed approach and discusses future improvements.

IOPES project
IOPES is a two-year project co-funded by the European Commission involving 7 partners from 5 different European countries. IOPES targets at strengthening the preparedness of emergency personnel by making them more responsive to disasters.
IOPES seeks to improve an already operational Emergency Management System (EMS) -software tool targeted at the handling of emergencies -by providing real-time updates of the position of the teams in the field. Nonetheless, IOPES is not targeted exclusively at improving this specific system but has been designed to interface to any other one by means of a standardized Application Programming Interface (API).
The ability to collect time-tagged positioning information -that may later on be related to specific, significant eventsfacilitates the post-even analysis of the disaster, opening a door to derive new strategies or procedures or the enhancement of these.
The project is funded by the Union Civil Protection Mechanism (UCPM) whose goal is to "improve the quality of EU response capacities" as stated in its Annual Work Programme (2019). Besides that, the IOPES is also fully aligned with priority 4 of the Sendai framework for disaster risk reduction 2015-2030, "Enhancing disaster preparedness for effective response". The project involves the combination of several technologies ( Figure  1) including RPAS-based fast mapping, emergency management, portable communications, and positioning technologies. IOPES aims to provide continuous, time-tagged information about the location of Civil Protection Emergency Teams (CPET), either indoors or outdoors.
This paper does not cover all the technologies involved in IOPES but only those related to the reliable indoor / outdoor positioning of emergency staff.

Hardware
The already developed prototype is a portable, low weight positioning device made of Commercial Off-The-Shelf (COTS) hardware components, mounted on a helmet but also able to be boarded as a drone payload. The positioning sensors used to build the IOPES solution are a GNSS receiver (Drotek DP0803 GNSS module (Drotek, 2021) that includes the U-BLOX NEO-M9N chip (ublox, 2021)), a stereo camera (Intel RealSense T265), (Intel, 2021), which also includes an IMU and a magnetometer (STMicroelectronics LIS3MDL) (ST, 2021). The GNSS receiver has been selected because it is able to provide meter-level accuracy and receive signals from up to four different single-band GNSS constellations. With this GNSS receiver the system has reliable GNSS position coordinates even in weak GNSS conditions such as urban canyons, exploiting its capability to select the best signals.
The T265 includes a built-in tracking proprietary algorithm running on board. It combines the measurements coming from the IMU and the images produced by the stereo fisheye cameras to provide, by means of visual-inertial odometry/SLAM, a seamless indoor/outdoor solution (Tsykunov, 2020); it provides the current position and orientation with an output rate of 200Hz. The magnetometer delivers raw magnetometer measurements with an output rate of 40 Hz, used to derive absolute heading.
The output frequency of the fused data (GNSS+camera) is 1 Hz. All these components (both hardware and software) are running on a lightweight computer (Raspberry Pi 4 Model B) (Raspberry, 2021), mounted on a helmet designed for work at height and rescue (Petzl Vertex), and powered with a 10 Ah power bank (Anker PowerCore Slim).
The Raspberry Pi is a system on a chip (SOC) with low power requirements that complete and integrate the set of components making the system. Its light weight ensure that it is not a nuisance for its wearers. Its task is to provide the necessary computing resources and storage capacity. Obviously, the low consumption requirements lead to longer operational times, thus reducing the need to replace batteries so often.
From the computing power standpoint, a powerful Graphical Processing Unit (GPU) is not needed, since the computations involved in the visual-inertial odometry solution are performed by the camera device itself. The system also includes a communications module, a 4G USB dongle with a SIM card (Huawei E3372). Additional features such as a headlamp can easily be attached to the prototype. All the components mounted on the helmet have been installed without modifying its structure to keep the helmet's safety standards ( Figure 2).

Sensor fusion SW approach
The cornerstone of the approach presented in this paper is a data fusion algorithm that relates GNSS and camera-based positions providing a single trajectory, regardless of whether it originated indoors, outdoors, or both. The flowchart of the algorithm is detailed in Figure 2. The flowchart is an updated version of the one presented in (Angelats et al, 2020) including additional details on how to derive a combined trajectory.
In areas with low or denied GNSS availability the camera-based tracking system is the main source to provide the positioning solution; conversely, the GNSS positions are used when it is available A common temporal reference frame is necessary to deal with data coming from these two sources -the internal clock of the SOC is enough for the purposes of the project. The GNSS solution, in friendly GNSS areas, is also used to convert the positions provided by the visual-inertial odometry (VIO) from local to global coordinates. If the portable positioning device starts to acquire data in an indoor area, the system is designed to store the VIO positions (in local coordinates) and the three attitude angles in the internal disk till absolute positioning is available. When the positioning device moves to an outdoor area and the number of GNSS satellites allow to provide a set of reliable GNSS based positions, a local to global transformation can be estimated, and, consequently, VIO positions in global coordinates can be provided from this moment. The criterion to consider a GNSS position as valid is that the horizontal and vertical errors provided by the GNSS receiver are below a pre-defined threshold (such as 10 m), thus indicating a good GNSS satellite geometry and a good position estimation. The horizontal and vertical errors are the receiver's estimation of one-sigma horizontal and vertical errors.
To estimate the local to global transformation, at least two valid consecutive GNSS positions in different locations are needed to have not only the transformed positions themselves but also to estimate the heading angle derived from GNSS. Alternatively, absolute heading can be derived from the raw measurements provided by the magnetometer. The criterion to select either heading source is defined by the user. In our approach pitch and roll angles are assumed to be close to zero although this may introduce some positioning errors. Then, a rotation matrix can be computed; afterwards, a global VIO position, the lever arm offset between the GNSS and VIO devices and the local VIO coordinates can be estimated using this rotation matrix.
Although not currently implemented, the GNSS-based positions will be used to model positions. This track can be used during the management of the emergency or once it is over to perform a post-mortem analysis.

Operational aspects
Ergonomics rule the operational aspects of the IOPES portable device. This means there must be no noticeable difference from the user's standpoint concerning how the system is operated either in indoors or outdoors environment.
Positions and time tags are computed according to the procedure described in section 2.3 and sent to the Emergency Management System so the managers may track the team working in the field -note that the components in the IOPES device as well as the remaining infrastructure required to make communications possible are not described in this paper, although these are an integral part of the project.
Under some environmental conditions, however, it will not be possible to compute any positions at all -examples of such adverse situations are dust or no lightning; these are limitations of the technology used for indoor positioning and thus affect the performance of the IOPES portable device.

Dataset description
A series of campaigns were carried out to validate the performance of the positioning sensors (visual-inertial odometry and GNSS sensors) and that of the overall system. The campaigns were done at the premises of CTTC in the Parc Mediterrani de la Tecnologia (Castelldefels, Spain) and their surroundings (Figures 4 and 5) and at Garraf town and surroundings ( Figure 6). Both scenarios are characterized by including indoor spaces, clear-sky spaces, and areas with strong multipath conditions.
The performance analysis of the system has been carried out considering several environmental conditions: distance to closest targets (outdoor-clear sky/ outdoor-low GNSS availability/indoor), system dynamics (kinematic/almost static/ static), environmental texture (no texture/texture) and lightning conditions (bright / dark). Four different routes (two per location) were defined and carried out combining the different parameters stated above. The first route, "Sa Falconera", took place in the surroundings of Garraf including a walk near a cliff, and a tunnel. The second route, "Garraf Town", is a walk inside the Garraf town, with narrow streets and ending inside a house with four different rooms in the same floor. Third route, "PMT-LAB", is a walk through the campus including an area with strong multipath and a walk inside the positioning lab of CTTC. Last route "PMT-buildings" is an extension of the previous one but covering all the corridors and main spaces of three floors of the CTTC-B4 building and also two different floors in CTTC-B6 building.

Performance analysis methodology
In order to assess the performance of the system, the estimated positions in the most relevant areas were analysed. Performance indicators are computed using predicted positions from equivalent environments like outdoor clear-sky, indoor or urban corridors. Then, for each of those subsets the mean accuracy and precision were computed; the references were the coordinates provided by PNOA (Plan Nacional Ortofotografía Aérea) maps (outdoors) and plans of the building (indoors). According to specifications (PNOA, 2021), the planimetric accuracy of the PNOA orthophoto should be better than 0.5 m (Root Mean Square Error) and 1 m for the height component.
For the specific case of the "CTTC positioning lab" (indoors), the coordinates of an easily identifiable set of points in the floor were computed by means of other reliable positioning techniques. Such coordinates constituted the reference for this use case. Later, a walk using the IOPES portable device was recorded, and its output coordinates compared with the aforesaid reference data set.   This research does not consider the error estimation provided by the GNSS as performance indicator since it has been observed that these values are too pessimistic in all trajectories.

Results and discussion
The system output for the "Sa Falconera" route is shown in Figure 7. The system estimates a set of global positions (depicted as dots), and just for clarification, each global position has been coloured red or green depending on the sensor used for its generation (green dots for GNSS estimated positions, red dots for camera-based ones). The summarized performance results for "Sa Falconera" test site are shown in Table 1. The results confirm the good performance of the system under "outdoor clear sky" conditions. Performance gets slightly worse when the system is outdoors surrounded by thick vegetation.
When the results are produced by the camera-based sensor the precision is good while the accuracy goes to 3 m.
The system output for the "Garraf Town" route is shown in Figure 8. The summarized performance results for "Garraf Town" test site are shown in Table 2. In this case the use of the camera-based sensor is higher due to the bad quality of the GNSS signal inside the narrow streets of the village. As in "Sa Falconera" the results under "outdoor clear sky" conditions are good. For most of the outdoor urban section of the route and for the indoor section the camera-based sensor has performed with good precision and 3 meters of accuracy.
The system output for the "CTTC-LAB" route is shown in Figure 9. The summarized performance results for "CTTC-LAB" test site are shown in Table 3. The most critical section of this route is the ~50 meters walk in a zone where the GNSS receiver suffers from multipath effects. In this zone the performance of the system gets worse, with a mean precision of 1 meter and a mean accuracy of 4 meters. The rest of the route is estimated with the same precision and accuracy performance of "Sa Falconera" and "Garraf Town".
The system output for the "PMT-Buildings" route is shown in Figure 10. The summarized performance results for "PMT-Buildings" test site are shown in Table 4. In this route the system is tested extensively indoors, under different lightning and space conditions. The outdoor results are similar to the previous study cases. The indoor results show no differences in terms of performance when the lightning conditions are changed. However, it can be observed an important worsening of the performance when the system is tested in big diaphanous spaces (due to the lack of features to be tracked by the camera). In these conditions the mean precision goes to 2 meters and the mean accuracy to 5 meters.    The results obtained in the different scenarios demonstrate the operability of the idea of the IOPES project. The precision and accuracy obtained for the different scenarios partially cover the user requirements of the IOPES project. However, the authors strongly believe that the user requirements for outdoors can be easily achieved improving the quality of the GNSS receiver (for example using a multipath-rejecting GNSS antenna and receiver). The indoor performance can be trickier to improve; the bad quality of the results in indoor big spaces seems to be due to the technology used and hardly improvable; however, the authors have found that the accuracy of the indoor solution in the rest of the cases can be improved by selecting properly the point where the solution changes from outdoor (and thus from the GNSS sensor) to indoor. Taking this point into account, more complex algorithms for detecting the outdoor/indoor transition could lead into an improvement of the indoor system performance.

CONCLUSION AND FURTHER RESEARCH
This paper presents the first results for a new, portable, lightweight, unobtrusive positioning device which, combining GNSS and VIO, is offering seamless indoors and outdoors positioning in the context of emergency and disaster management applications and the monitoring of the teams involved in these events.
Such a device is expected to operate in many different scenarios -such as open sky, tunnels, rooms inside building -under different environmental conditions -such us lack of lightning or texture -and subject to changes concerning the elements making the scenarios themselves -as, for instance, due to the presence of moving objects such as people or vehicles.
The expectations concerning the positioning parameters defining the performance of the IOPES device were, literally, "to be able to tell apart both the room and floor where the individual being tracked was". This was a requirement specific to indoor positioning, which the authors translated to more measurable magnitudes, that is, expecting an accuracy of about 1-2 meters and precision in the range 30-50 cm (see section 1).
The results commented in section 3.3 show that the requirement concerning accuracy has not been achieved, going up to 4 meters in one of the use cases. The reason explaining this result is directly related to the accuracy obtained by the GNSS receiver itself in the aforesaid scenarios; the overall accuracy of the IOPES device directly depends on this magnitude. It is reasonable to expect, therefore, that this accuracy will be improved when working in more favourable situations. Precision, on its side, meets the expectations stated in section 1.
Nonetheless, accuracy and precision are either suitable or not depending on the target application and, specially, on the users involved in the exploitation of such application. It is worth to note that at least one of the users involved in the IOPES project finds these results very interesting, and, consequently, the IOPES device useful for their purposes. The opinion of the rest of the users is still unknown to us, but they will have the opportunity to express it soon, during the first demonstration of the project. The authors are convinced, however, that such opinions will go in the same positive line than the first one.
At any rate, the work on the IOPES device is not yet over. The results presented in this work correspond to the very first assembled prototype; at the moment of writing this paper, more than six months remain to improve the current results and make possible a second prototype including enhancements of diverse kinds. The work to come will concentrate in making the prototype more robust and performant. The foreseen lines of work are the implementation of the correction of the drift when working in VIO mode and a better mechanism to detect indoorsoutdoors (and vice-versa) transitions to reduce the time to switch between to two positioning technologies -and thus, reduce errors. The authors expect that both lines of work will serve to improve the accuracy of the system.