INTERACTIVE VIDEOS AS GEOSPATIAL INTERFACES: A CASE STUDY FOR REGIONAL PROMOTION

In this paper we discuss the potential of interactive video as an interface to geospatial information, and demonstrate our findings in a case study driven by goals of promoting a region for tourism and for preventing brain drain. Use of video is controversial due to various technical and human-centric debates; on the one hand, they offer unique qualitative insights as they enable a certain level of immersion via walk-throughs and fly-throughs and they are considered attractive by viewers, on the other hand, they are complex because they contain unfiltered (possibly too much) information that may be irrelevant to the goals of the users, and require careful consideration from computational and bandwidth concerns. We examine these arguments from the literature briefly, and demonstrate our case study in which we have embedded and overlain statistical and other local information interactively over drone footage, and measured levels of usability and user engagement using standardized scales. Our implementation and design choices are detailed in the article. Our small user experiment (n=6) suggests high levels of usability, desirability and engagement by our participants, leading to the hypothesis that video is an attractive medium and indeed has strong potential for regional promotion, given that the computational and cognitive issues are taken into account. 1* Corresponding author: arzu.coltekin@fhnw.ch


INTRODUCTION AND BACKGROUND
Videos are a rich source of information containing realistic geospatial and temporal context. With the shrinking of modern sensors both in size and in price, we have access to more and more video footage from webcams, surveillance cameras, drone footage, etc. The potential of videos have been previously considered in geospatial sciences mostly as a data source in e.g., mobile mapping (Mills et al., 2010), or as a complementary feature in geographic information systems (GIS) (Lewis et al., 2011;Kuhn et al., 2011). There has also been efforts in spatially indexing videos to make them searchable for geospatial content, or automatically generating tags for them (Ay et al., 2008;Yin et al., 2015), also often with the intention to integrate them into GIS (Milosavljević et al., 2016). Furthermore, with the developments in computer vision and machine learning, video analytics is becoming easier and more exciting (Chemodanov et al., 2020). However, despite the increasing data availability and various use cases, videos are rarely examined or used as geospatial interfaces. There are several reasons for this. Video is computationally expensive and can create bandwidth problems (Çöltekin & Reichenbacher, 2011); or changing views ('animation') might not allow users to digest the information or make comparisons (Tversky et al. 2002, Russo et al., 2014. It is also understood that, "too much realism" can impair user performance in certain tasks (Thoresen et al., 2016, Krejtz et al., 2017, Smallmann & John, 2005. On the other hand, people find photorealistic and 'quasi 3D' representations attractive (e.g., Boer et al., 2013, Çöltekin et al., 2015, Lokka et al., 2020, and the value of 'fun' should not be underestimated in contexts where user engagement is important, such as in education or promotion, as in the case study featured in this paper. We believe with proper interactive controls, wellconsidered annotations, and ability to switch between (or combine) video and maps or other visualizations; the potential of video footage as interfaces to a 'digital earth' should be studied further. In this paper, we present a case study in which we examine the use of annotated videos as interactive geospatial interfaces to enable users to explore and discover a region. A specific goal in the case study is to attempt making the region attractive to bring in more tourism and prevent brain drain. Specifically, we examine the following questions: -What are the benefits and limitations of using videos, specifically drone footage, as an interactive interface for geospatial exploration on the web?
-What design considerations are important in such a video interface given the goal is to promote a region for tourism and prevent brain drain?
-What are the usability issues to consider in a video interface? Are such interfaces engaging for the users?

User centered design
Our user centered design approach consisted of a research phase and iterative user-testing cycles throughout the implementation phase. In the research phase, we surveyed alternative visualizations and identified a target user group for our application, employing standard user experience methods such as design thinking (Plattner et al., 2013), and working with user stories and personas (Allen & Chudley, 2012).

Implementation of the prototype
To implement our prototype(s), we utilized a set of open source libraries, specifically, javascript and CSS for animations, React.js for the web application, D3.js for the interactive visualizations, thus SVG for our vector graphics (Table 1). An overview of web-page's architecture is sown in Figure 1.

User study
Study design and participants. The primary aim of the user study (n=5, 2 women, 3 men, age range 24-45, professionals) was to examine usability of the prototype, and get first insights on whether our concept of using drone footage in combination with layered interactive elements encourages decisions visiting the region, or even living and working there.
Given the above, we were keen to understand whether the concept successfully engages the visitors of the webpage, offering them a rare web experience, thus adhering to the web page longer, and consequently discovering not only facts but also e.g., attractive living spaces or possible employers in this region. For this purpose, an exploratory testing method was selected (rather than a hypothesis testing approach). This allowed us to observe participants' opinions, emotional expressions and personal feelings.
Subsequently, we evaluated the overall user satisfaction based on the System Usability Scale (SUS) (Brooke, 1996) and user engagement levels based on User Engagement Scale Short Form (UES-SF) (O'Brien et al., 2018). Both are standardized scales, we adapted them slightly more to our context, and asked the following questions (Table 2a, 2b). Table 2a. SUS questions used in the study (modified from Brooke, 1996) 1. I found the website unnecessarily complex 2. I thought the website was easy to use 3. I think I would like to use the website frequently 4. I felt lost on the website 5. I think that I would need instructions on how to use the website 6. I thought there was too much inconsistency in this website 7. I could not find the information I wanted 8. I imagine that most people would learn to use the website very quickly 9 I felt very confident using the website 10. I found the various functions in this system were well integrated Procedure. Once participants arrived at the laboratory, we welcomed them, explained them briefly what is expected of them, have them sign the consent forms, and gave them a brief training with the system to bring everyone at the same level.
Once the study began, we gave participants multiple tasks to enable them to experience the various aspects of the implementation. These included three scenarios that were prepared for three personas, 1) Explore the intro, 2) Explore the city, 3) Filter and discover. More specific instructions were given partially verbally and partially on the computer screen.
After experiencing these scenarios with the support of the webpage, participants answered the following questions: -Explore the first three screens of the application and discover what you can learn about the canton. How do you feel about further exploration after you have seen these three pages?
-Imagine you are a construction expert and are looking for a job in Aarau. Hence, you are only interested to see more information about companies in this field. What are your next steps?
-Imagine you are now interested in the region of Baden. Start the adventure from the third slide. You are free to explore the area. How do you feel about your experience?
Following these we asked them further open-ended interview questions to gauge their attitudes further on e.g., how much time they think they would spend on the webpage, if they would use the page again, for what purpose it would be, and what new information they gathered from the experience. At the end, they answered the SUS and UES-SF questions. Participation was voluntary and there were no specific benefits or drawbacks for the participants. Throughout the experiment, we took notes on how many times participants asked for help, and number of positive/negative comments as well as nonverbal reactions such as smile/laugh or frown.

Interactive prototype
Since a key goal in the study was to help prevent brain-drain besides tourism purposes, we identified skilled workers as a target group. Thus, in six iterative design cycles, we collected skilled workers' reactions to our designs qualitatively to create a design prototype. A specific design goal was to create an attractive interface on which certain elements could be highlighted. After the research phase and initial user feedback from the abovementioned iterations, we decided to use video, specifically, drone footage as an annotated, interactive interface (Fig. 2, Figure 3a, and 3b) and also implemented optional 'fun' quizzes about certain features. Oblique views from the drone footage provide both overview and context, enabling a rare and relatively novel experience, as common geospatial interfaces are typically aerial or first-person view. In essence, our design follows "overview first, zoom-and-filter, details on demand' principle (Shneiderman, 2003). In our prototype, video sequences feature a few selected cities/towns, and stop automatically at so-called breakpoints to switch the control to the user. Viewers can explore this spot, or continue to the next breakpoint. Selected scene elements are highlighted with animated lines to draw viewer's attention to the objects that contain more information. Further information can be queried on demand, i.e., one can click on the interactive button to request more information triggers data visualizations ( Figure  3a), text, icons, etc. about that specific POI (Figure 3b). Users can dismiss the overlay with a back button, and can explore other interactive buttons or continue to the next breakpoint.

Usability and user engagement
All participants were able to complete all tasks successfully, thus we can surmise that the interactive video prototype effectively facilitates exploration. Importantly for the goals of this study, participants were satisfied with their experience: Combined SUS score was 87.5/100. In SUS, above 68 is considered 'usable' and 80.3 or higher is considered an "A" (Brooke, 1996), and visual elements and website design were rated highly engaging in UES-SF (Figure 4). For example, all participants found the website aesthetically appealing, and they all liked the images and graphics (Figure 5), had great experience and fun and, importantly, they stated that they would return to the website out of curiosity and would want to spend a considerable amount of time exploring it, varying between 15 min to over 30 mins. Moreover, participant anticipated that the website would have a positive influence on the decisions of whether to visit the region or not.
Along with active questions in UES-SF and in the interviews, we observed participant reactions as they used the web page and noted down the number of times they have offered positive and negative comments, or other non-verbal expressions (Table 3). These observations suggest that the overall positive comments including non-verbal feedback is more than triple as the negative ones. The positive skew is obvious in user engagement scale ( Figure  4). Specifically, the ratings for "The website was aesthetically appealing" and "I liked the graphics and images used on the website" were unanimously 5 by all participants, indicating a strong element of pleasure. Also, "The website was attractive" and "I would return to this website out of curiosity" received 4x 5 and 1x 4 ratings, "The exploration experience was fun" statement received 3x 5 and 2x 4 ratings, and "I would recommend this website to my friends" and "The website can have a large influence on my decision on whether to visit the region" were rated 3x 5 and 2x 4. All other dimensions are also rated overall positively, with minor hesitations.
While the results of the user experience evaluation concerning engagement were mostly positive, the usability part of the testing revealed some improvement potential: the initial idea of the concept was presenting the video sequences as a continuous movie to the participants to provide an uninterrupted experience. The testing session revealed that all participants wished to have more control over the video, and the auto-continuation was rather perceived as an unpleasant element. Therefore, another iteration was made to introduce a new interactive component giving the users the possibility to continue to the next sequences according to their choice. This should ultimately result to better usability: the users will not be surprised anymore that another video sequence is suddenly triggered without any prior notice or interaction.
Interviews. Some of the comments made during the openended questions and general feedback were very encouraging. Some example positive remarks include: "Very intuitive"; "I think the animations are beautiful. I have never seen something similar so far, it is quite unique."; "I like that I choose what I want to see. I have the control over the content."; "I like the animations a lot to be honest."; and "The quizzes are funny". Some more specific to the interface were "The overlay windows with more information are really nice."; "The filters  (Table 2) on attractiveness, aesthetic appeal, visual complexity, adherence, reuse, the influence of the website experience on their decisions to visit the region. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B4-2021 XXIV ISPRS Congress (2021 edition) are a good idea."; "I like the slide-in window, it is like a tip what to do in the region, the visual language is appealing and information relevant.". Some of the remarks contained specific, design-related feedback which was very useful; some included comments on the readability of labels due to font size or background color, the fact that some of the buttons were not interactive (participant thought they were all interactive), interpretability of the icons (specifically, participant did not understand the drone icon), and the temporal behavior of the web page ('The timeline confused me, I thought I am already one point ahead").

DISCUSSION AND CONCLUSIONS
This paper provides a brief but critical look at the potential of videos as interfaces for geospatial exploration, specifically with the goal to promote a region for tourism and employment opportunities. The initial observations (as presented above) show that videos have the potential to be usable and useful from a user-centric perspective.
Even though we have a small group of participants (n=5), this is a sufficient number for usability studies (Nielsen, 2012), and our results regarding the usability of the prototype should not be affected too much by adding more users. We did discover some usability issues and tweaked the interface design based on the participant feedback on the specifics regarding font, color, marking interactive buttons differently than those that were static etc., and marking the timeline so that user always has information on the temporal context. Our other findings, specifically, the UES-SF results are preliminary (there is no evidence that five participants are enough for this test, as opposed to usability testing). However, they confirm the previous findings in literature suggesting that photorealism is attractive and/or engaging (Lokka et al., 2020;Smallman & John 2005;Çöltekin et al., 2015), we can see a unanimous agreement among our users that interactive drone videos are attractive as interfaces.
Depending on the project goal, there may be issues of cognitive load, and we have to keep the naïve realism proposition in mind, i.e., employing realism 'just because' people find it attractive is not always a valid argument, because, in some tasks, realism can hurt human performance with visuospatial displays (Smallman & John, 2005;Çöltekin et al., 2017Çöltekin et al., , Lokka & Çöltekin, 2019. However, we believe the highlighting over the video/imagery as we did might help alleviate performance issues (hypothesis to be tested), and importantly, if the goal is promotion, engagement (and in some cases, memorability, e.g., Borkin et al., 2013, Lokka & Çöltekin 2019, visual realism can indeed be a good (even great, if our five participants in this study is an indication) option.
When working with heavy imagery and/or video, another limitation may stem from computational issues, primarily bandwidth management, which requires optimizing the data streaming, though there are some solutions for this such as space-variant image coding, or foveated streaming (Çöltekin 2009(Çöltekin , Çöltekin & Reichenbacher, 2011. With future research investigating these potential issues, we believe using video footage as an interactive interface is an attractive, promising alternative to the more traditional geospatial interfaces, especially if the goal is to promote e.g., a geographic region.