ON THE USABILITY AND USEFULNESS OF 3 D ( GEO ) VISUALIZATIONS-- A FOCUS ON VIRTUAL REALITY ENVIRONMENTS

Whether and when should we show data in 3D is an on-going debate in communities conducting visualization research. A strong opposition exists in the information visualization (Infovis) community, and seemingly unnecessary/unwarranted use of 3D, e.g., in plots, bar or pie charts, is heavily criticized. The scientific visualization (Scivis) community, on the other hand, is more supportive of the use of 3D as it allows ‘seeing’ invisible phenomena, or designing and printing things that are used in e.g., surgeries, educational settings etc. Geographic visualization (Geovis) stands between the Infovis and Scivis communities. In geographic information science, most visuo-spatial analyses have been sufficiently conducted in 2D or 2.5D, including analyses related to terrain and much of the urban phenomena. On the other hand, there has always been a strong interest in 3D, with similar motivations as in Scivis community. Among many types of 3D visualizations, a popular one that is exploited both for visual analysis and visualization is the highly realistic (geo)virtual environments. Such environments may be engaging and memorable for the viewers because they offer highly immersive experiences. However, it is not yet well-established if we should opt to show the data in 3D; and if yes, a) what type of 3D we should use, b) for what task types, and c) for whom. In this paper, we identify some of the central arguments for and against the use of 3D visualizations around these three considerations in a concise interdisciplinary literature review.


INTRODUCTION
Creation of 3D visualizations has largely been driven by rapidly developing technology rather than cognitive and perceptual theories and/or user-centered thinking (Çöltekin & Haggren, 2000;Fuhrmann et al., 2005).As the processing power and memory capacity of the computers increased, and software grew more sophisticated; the use of 3D graphics in many domains increased --simply because it was now feasible to create them.Thus, the use of 3D in visualization appears to be common today, even though there is very little understanding and awareness if 3D visualizations work as intended for the aimed audiences and tasks.
Many of the arguments in this vein remain theoretical, sometimes anecdotal and/or based on introspection of the scientists themselves.However, there is also some evidence from empirical studies, and based on these, it seems that the answer to whether we should use 3D or whether it is 'any good', is 'it depends' (Huk, 2006;Shepherd, 2008;Tory et al., 2006).In other words, as various forms of 3D have been growing popular (e.g., in media or in scientific publications), we also started hearing some critical evidence that cautions us to think before we go for the 'cool' 3D effects (Hegarty et al., 2009).Geographic information scientists have also shown interest in this conversation in recent years publishing evidence for and against the use of 3D through empirical experiments (Zanola et al., 2009).

CONCEPTUAL ORGANIZATION OF CURRENT EMPIRICAL STUDIES
Conceptually, to tease apart the evidence for and against the use of 3D, we propose categorizing the results from the existing empirical studies based on three fundamental dimensions 1 that are critical in user studies using visualizations as stimuli (Çöltekin, 2015;Tory, 2014): 1) Visualizations typewhen you hear 3D, what comes to your mind?There are many 3D visualization types that vary from very abstract simple plots to fully immersive virtual reality environments.When we consider an empirical result, we should pay attention to what type of 3D has been tested.Furthermore, as in all visualizations, related design choices (Oh et al., 2011) such as interactivity (e.g., Amini et al., 2014), use of color (e.g., Brychtová & Çöltekin, 2016), or degree of realism e.g., phototextures, virtual environments (e.g., Boér et al, 2013;Smallman & Cook, 2011).
2) Task type -Similarly, as in all user studies, the context in which the study was conducted is important to identify.Here it is important to distinguish between task and instruction.However, they are equally important, results from one empirical study might not apply to another one if the task (i.e., what goal was) and the instructions (i.e., what the participant was asked to do) are different.Even small details matter as one can 'prime' the participants based on the instructions (Martin, 2008).To do this systematically, first of all a new (geo/spatial) task taxonomy, expanded on previous efforts is needed (e.g., Carter, 2005;Knapp, 1995).Once there is a relevant, up-to-date task taxonomy is prepared; studies should be classified (and conducted) accordingly.
3) 'User type', i.e., individual and group differences, i.e., human perceptual and cognitive abilities are important (e.g., Slocum et al., 2001).Not only we should study if 'this type of 3D is good for task type x' (for what) but we should study if 'this type of 3D is good for task type x and participant type y' (for whom).Literature abounds with examples that expertise, i.e., education, experience, previous exposure (e.g., Çöltekin et al., 2010), spatial abilities (Liben & Downs, 1989;Huk, 2006), visual abilities (Fukuda et al., 2010), age (Schnürer et al., 2015), and possibly other characteristics, such as lack of sleep (Kong et al., 2011), or if the alphabet is pictorial/iconic, or if one reads and writes from left to right, or if one conventionally uses certain graphical designs, etc.) all have an influence on whether we benefit from working different kinds of graphics (including 3D), or not.
In terms of how to measure the relevance and fitness of 3D, in many user studies, we see performance (usually accuracy and speed of task execution) as the main criteria.While clearly very important in many tasks, there are possibly other important considerations too.For example, if 3D is perceived as 'cool' (i.e., attractive, interesting), it could be more engaging, thus could have a value in situations where grabbing attention or engaging people are important.As a larger goal, we plan to look for patterns in the user studies based on the three dimensions described above, organize the findings in the literature systematically based on the reported outcomes in empirical studies not only based on performance, but also on preference, confidence, attractiveness, and levels of presence.In this paper, we present broad categorizations of existing 3D visualization types, related tasks, and participant characteristicsand based on these we identify some of the central arguments for and against the use of 3D in visualization2 .

3D Visualization types: Not all are created equal
A current example of technology-driven excitement about 3D visualizations is the (re)popularization of a specific type of 3D, i.e., the highly realistic immersive virtual reality environments (VE).3Seemingly, visual realism is an important part of the discourse on 3D.However, it is necessary to note that not all 3D are realistic, nor all realistic visualizations are 3D.In Fig. 1, we present a non-comprehensive categorization of a set of example geovisualizations based on realism and dimensionality.
On the other hand, level of realism can be studied with immersivenes, showing that not all realistic representations are immersive.Thus, focusing on the 3D visualizations alone, here we propose a rough categorization based on levels of visual realism and levels of immersiveness (Fig. 2).Highly immersive and highly realistic visualizations are a specific focus in this paper, because they are still rather effort and cost intensive to create, and because of their oscillating popularity through the history, and the current rise of this popularity.This popularity is evident as modern head-mounted displays of decent visual quality (yet considerably less expensive than before) are currently in the consumer market, and even 'cardboard' versions of these are available, enabling immersive stereo viewing using mobile phones (Spilka, 2015).
Such developments increase the accessibility and ubiquity of VE systems, which were once prohibitively expensive, and clumsier to use.VEs differ from other types of 3D, as they mimic reality and are typically designed to be highly immersive, have high information intensity (realism), high interactivity and often feature intelligent objects (MacEachren et al., 1999).However, by definition, VEs feature 3D visualizations in them.
In that sense, VEs can be seen 'type' of 3D visualization, while there are many ways to visualize 3D information.

Tasks types
An attempt to categorize 3D visualization types (e.g., as in Fig. 2 and usability.Of course, categorization efforts of 3D visualization types could also be approached based on other factors, e.g., levels of interactivity, based on the task types they are designed for, or domains of use.Today, diverse professional groups use different kinds of 3D visualizations, spanning from highly realistic virtual-reality type 3D renderings to abstract 3D plots (Fig. 2), for a large number of tasks such as in education, virtual tourism or planning (Biljecki et al., 2015).We see 3D visualizations in practical applications, e.g., advertisement industries, civil engineering, architecture, urban design; in journalism, e.g., political news, weather reports; as well as in the scientific world, e.g., geography, medicine, engineering; or in fine arts (e.g., Borkin, et al., 2007;Neuenschwander et al., 2014;Wood, et al., 2005;Yeung, 2011).
For example, in urban design (e.g., Fig. 3), one might hypothesize that the closer the visualization is to 'reality', the easier it may be for the urban designers as users to comprehend what is being proposed and make decisions (Hayek et al., 2011;Herbert & Chen, 2014).
Figure 3. Viewing a specific type of zoning plan (atypical) in 2D (left) or in 3D (right) could make a difference in participant performance as well as attitudes.Note that both the dimensionality and the viewing perspective changes here, and therefore different information are shown.
On the other hand, even though it might present certain advantages in some cases, such as revealing information that may not have been otherwise visible (Li et al., 2010), or people might prefer it for a set of tasks (Çöltekin, et al., 2015) there is strong evidence that for reading plots (Dall'Acqua et al., 2013), or detecting anomalies (Borkin et al., 2011), 3D visualizations might rather hinder their users than help them.Various reasons have been proposed for the causes of the failures with 3D visualizations.For example, the scale variation on 3D displays makes judging distances and areas harder, occlusion sometimes removes relevant information from the display, and, displaying 'more information' (such as in more realistic visualizations) is likely to induce information overload (Harrower & Sheesley, 2005;MacEachren, 1995;Seipel, 2013;Shepherd, 2008).

Human factors -'participant type' or individual and group differences
15 years ago, Slocum et al. (2001) published their seminal paper on cognitive and usability issues, and have stated that the virtual reality environments 'fundamentally change our traditional way of acquiring spatial knowledge' (Slocum et al., 2001, p. 62).Years later, we see that virtual environments remain relevant, yet from a cognitive and usability perspective, information overload can be a real threat to user performance with realistic visualizations (such as VEs).VEs have high information intensity, as they mimic reality and attempt presenting the visual environment with as high fidelity as possible instead of a meaningful summary with highlighted information.While attempts to manage the level of detail with modern means such as gaze-contingent displays have been proposed (Duchowski & Çöltekin, 2007;Çöltekin, 2009;Bektas et al., 2015), these are experimental and certain human factors issues hinder their use.For example, most VEs are based on stereoscopic visualizations and, according to Ware (2004), as much as 20% of the population is not able to see in stereo 3D.Additionally, it is well-documented that stereoscopic displays can cause discomfort such as simulator sickness or eye strain (Lambooij et al., 2009).On the other hand, a comprehensive review of empirical studies regarding stereo 3D suggested performance improvement in 60% of the cases considered, 25% of the cases it was a 'tie' between mono and stereo, and 15% was mixed/unclear (McIntire et al., 2014).McIntire et al. (2014) state that stereo displays clearly offer benefits in depth-related spatial tasks, however, they caution that 25-50% of the population expresses some discomfort with stereo displays and they should be designed carefully.There are also other basic perceptual problems with other kinds of 3D, e.g., we may not be able to perceive depth correctly if the light source is in the 'wrong' position (Bernabé Poveda & Çöltekin, 2014;Biland & Çöltekin, 2016;Imhof, 1967).
Besides the perceptual problems, individual differences are expressed in based on cognitive abilities.As mentioned earlier, possibly because they have an advantage in terms of coping with the information overload, high-spatial people might be benefitting more from working with certain types of 3D in certain contexts (Huk, 2006).Though the opposite has also been suggested, with the argument that 3D might be more 'intuitive', i.e. resembling the real world objects more than the 2D alternatives should be easier to recognize, thus low-spatial people might benefit more (Rautenbach, et al., 2015;Shepherd, 2008).Currently there is no established understanding of exactly in which situations these arguments might apply (to which kind of 3D, what design choices, and in relation to which task).In addition to spatial and cognitive abilities, there is also evidence that previous exposure (familiarity, education, expertise) affects performance with 3D (e.g., Green & Bavelier, 2007;Harrington, 2011;Sungur & Boduroglu, 2012).Such evidence is especially interesting because it suggests that some of the individual differences are malleable (Uttal et al., 2012), thus before we attempt designing for a specific group, or personalize the design, we might want to consider if there are possibilities to intervene trough training.

CONCLUDING THOUGHTS AND FUTURE WORK
In this paper, we provided a concise first review of the findings in the existing empirical studies on 3D visualizations.Our initial review of the literature revealed some specific aspects of working with 3D visualizations that are worth considering.For example, some studies suggest that 'human recognizable' elements in a visualization might be helpful in how well we remember them (Borkin et al., 2013), which encourages us to further study the benefits of (photo)realistic 3D visualizations such as VEs.Of course, when we talk of memory, individual and group differences (as mentioned above) are especially important, because specific populations (e.g., based on age, spatial ability, and professional experience) might have different strategies and memory capacities (Lokka & Çöltekin, 2016).
In terms of tasks, our current review broadly suggests that tasks that require global visual information processing (i.e., studying the entire scene) such as in scene gist recognition (Loschky et al., 2010), object recognition or for remembering things, realistic (3D, VR) visualizations may offer advantages, while  (Gasper and Clore, 2012), and specifically for spatial memory (Brunyé et al., 2009).In terms of domains, realistic 3D representations such as VEs still hold promise in simulating real or fictional environments and provide immersive experiences in which one can safely conduct experiments under controlled conditions; experience future or past, explore under the oceans or visit Marsthus when designed well and used for the right purpose, 3D/VR may not only be useful but also be engaging and entertaining, and thus might have potential in serious games (with parts of 'edutainment', 'game-based learning', 'e-learning') (Mortara et al., 2013).
A logical next step is to expand this concise review to a comprehensive one and provide deeper categorizations with more evidence in terms of visualization types, design choices, task types, and user groups.Our overarching future goal is to organize the existing information and conduct further empirical studies to understand what specifically does the usefulness and usability of 3D visualizations depend oni.e., if certain 3D visualization types work or do not work well for certain task types and possibly certain user groups, and if they do or do not, why could that be.

Figure 1 .
Figure 1.A rough classification of an example set of popularly used geovisualization types (Boér et al., 2013).

Figure 2 .
Figure 2. A rough classification of 3D visualizations based on their levels of visual realism and immersiveness.
) could help better organize what we know about their utilityÇöltekin, A., Lokka, I.-E., & Zahner, M. (2016).On the usability and usefulness of 3D (geo)visualizations --A focus on virtual reality environments.In ISPRS -International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.Prague.
Krejtz et al. (2014) I.-E., & Zahner, M. (2016).On the usability and usefulness of 3D (geo)visualizations --A focus on virtual reality environments.In ISPRS -International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.Prague.for tasks that require local visual information processing (e.g., telling precisely if temperature is 2 or 3 degrees higher at point A than at point B) it may introduce problems.RecentlyKrejtz et al. (2014)have also proposed a similar 'global/local' distinction for measuring types of attention based on eye movements in spatial tasks using the terms ambient and focal attention, and earlier literature also distinguishes local and global visual information processing