IMMERSIVE WAYFINDING : VIRTUAL RECONSTRUCTION AND EYE-TRACKING FOR ORIENTATION STUDIES INSIDE COMPLEX ARCHITECTURE .

The study presents a process of digital simulation that aims to investigate the legibility of multiple spaces in a complex architecture through architectural survey, virtual reconstruction and 3D visualization in immersive environment. The collaboration between two research institutions, one Italian and the other Chinese, developed a reconstruction of a building in the campus of the Tsinghua University of Beijing using digital tools, in order to understand the behaviours during the fruition of that space. Digital simulation in the visibility theme guarantees the possibility of testing multiple configurations and showing the impacts of the different environmental hypothesis. Figure 1: Occupants behaviour data interpretation.


INTRODUCTION
The Cultural Revolution produced from computer technology, has empathized the concerts of model and image, through the determination of new connections and the subvention of fixed representative hierarchies, having as center the concept of "simulation" (Baudrillard, 1981).New representation techniques subvert the process inherent the statute of representation, the image spoliation from the apparent forms of lines and points (Klee, 1956;Kandinsky, 1926), which identifies the more complex geometry.While the model is always vests of what makes it more congruous to reality, empathizing the possibility of perceiving space in the virtuality of representation, intended as a result and no longer a process.Simulations' new perceptive potentialities that lead the observer in an "immersion" in its reality, live of "the vibration of the appearances, which is the genesis of things", as Merleau-Ponty asserts (Merleau-Ponty, 1962).Through a sequence of images, virtual simulations has to face the same issues faced by the study of reality, but with the possibility of being in a discretized space, easier to measure, which is not signed by the uniqueness and the "truth" of matter, but it is signed by the multiple possibilities of representing the parametrized form.In this contest, it is possible to see the immersive character to analyze today the built space, and evaluate, in an empiric form, the impact of the possible simulations.Through the study if the immaterial, the vision and neuronal stimuli (Katz, 1944), it is possible to investigate the discovery of space, because "man is mainly a visual animal, and more than the 50% of his brain neurons respond to this sensorial incoming" (Maffei, 2007).From this, it is possible to highlight which process leads to a reduction of virtual complexity in the abstraction of substantial geometry, which is the base of the design, in the contemporaneity of big data (Kitchin, 2014).At the base there is the hypothesis that our mind founds the structures of space in bi-dimensional synthesis, which are signed by the point and line, that in their one-dimensionality represent "the more eminent correct way of reproducing the perceived form" (Arnheim, 1954) in their "austere abstraction … and arid algebra" (Le Corbusier, 1923).It is then realized a coincidence between the synthesis of the thought and the schematization of the graphic act.The "materials" correspondence, the retina and the generic level, founds the process in the capture and reduction of the space into an image, identifying it in a projection (Gregory, 1966).The mind, in facts, organizes the bright stimulus through unitary forms and then in figures, in order to take the perceived image in recognizable typological families.These families are tight to categories and to a relation between identification and memory, a process that "organizes, completes and synthetizes the structure of particular optical images" (Arnheim, 1977).The scheme, in order to be analytic and descriptive, has to tend to a resemblance based on the correspondence of the substantial structural characteristics, as a map can renounce to every geographic particular to highlight the topological proprieties."The structure shows the formal proprieties for a relation system" (Norberg-Schulz, 1980) and it does not only sing out the discrete constituents, but it has to reconnect them in the unitary set of interdependent figures perceived in their holistic form (Kepes, 1944).The identification of a figurative structure does not come from a deduction, it does not deal with causes and effects, but it has a relational character, it conforms to the relation between the signs, which are homogenous by the figurative synthesis, and for this confrontable.From the equilibrium between sings, it is possible to highlight the "permanencies" (Poëte, 1929), peculiar elements that are not wear out over time but actually reinforced.These are places full of meaning (Jencks, 1969), deeply marked by the relation that conforms them as "significant signs" (Peirce, 1958) through the figurative structure.As Panofsky explains, "the aesthetic space and the theoretic space always translate the perceptive space by re-shaping it in a whole same hearing, which in the first case it appears symbolized and in the second one it appears in a logic way" (Panofsky, 1927) and, for this, it is possible to measure it.In this contest, Lynch introduces the concept of "legibility", where he correlates the identification to the solid structure of the image: "imaginability is that form, color or order that makes the building of mental images of the environment easier, so much that they can be identified, strongly structured and for this very useful" (Lynch 1960).It is possible to imagine a simple path, a correlation between figurative elements that become signals, which lead the way to know space.
It is then open the orientation theme: "the world can be organized around a series of focal points, it can be subdivided in regions and connected by memorable itineraries" (Lynch 1960).In their composition, points and lines are the protagonists of the figurative synthesis that creates the maps, the itineraries in the image, because in the perception "the uninterrupted activity conserves nervous energies.In its job the eye needs as much of action as of rest" (Kepes, 1944).Orientation and figurative structure begun their relation with the primordial man's tendency of being nomad, with the freedom of conquering space that happens through knowledge, the action that discovers an order (Lynch 1981).Bergson asserts: "The faculty of comprehending is an annex to the right to act, an always more precise adaptation, always more complex and ductile, about the knowledge of human being on their existence conditions" (Bergson, 1907).Orientation is in fact a mental motion, referenced with respect to the observer and based on polarity and meanings, which however includes a four-dimensional vision.Representation becomes a transposition of sensible elements in intellectual synthesis, "it is never a given space but a built space" (Panofsky, 1927), where from every point of the space equal construction are possible to be built.Le legibility of an architectonic space and the orientation to its inside are then themes that can be analyzed and evaluated, even with multiples simulations hypnotized in the virtual.The architectonic synthesis discloses the influence of connotative elements in reading space, but it still emerges the centrality of the act of seeing and participating to a formative process.It is a creative act" (Kepes, 1944).It is not an immediate perception, but a construction that lives the action.The journey is the best expedient to describe urban geography and it follows a logic sequence, as a speech about the place that explains its topics, as a discovery of the parts from the implicit vision of the whole.Orientation becomes then a narration of urban space, where the linear development of events is important as well as the story, which has its independent discovering path, founded on the late reconnection of the events.Space is then "timed, lived and socially benefited, in order to welcome and exalt the events" (Zevi, 1973).Its consequently heuristic becomes an exercise to project and create, a research of orientation, and for this an order based on the continuous movement between perception, memory and figuration, where the equilibrium process is proposed again, as a fruit of a groping research, experimenting different possibilities from where it will emerge the most significant one.

RELATED WORKS
The research in human perception topic has an unquestioned importance.Also in architecture, where the progressive digitalization of design, such as BIM design, provides always more reliable architecture models for always more reliable analysis and researches.Energetic and thermal model analysis are becoming a standard in the AEC industry.Technologies as virtual reality and eye-tracking as well are always more considered and not just for research purposes.Virtual reality, used at first for the exhibition of existing architectures, is now taking its space in the design process.Client-based design is one of the approaches that gets great benefits from the use of VR technology.Recently to present the design process of a kindergarten virtual reality resulted to be the tool that better fits the peculiar client's competence and visual perception.(Bakr, El Sayad, Shokralla Thomas, 2018).This device mainly studies human behaviour through the projection of photos and videos on a screen (Li et al., 2016).Lately, many experimentations began to use the eye-tracking combined with VR, and the obtained results are comparable to experimentations in a real contest (Meng and Zhang, 2012;Sharma et al., 2017;Vila et al., 2003).The potential of these technologies increases when combined between them and it brings new opportunities in the study of human perception.About the exploration of architecture in VR, a study regarding virtual simulation investigated how a given space is differently communicated using various representation methods.The comparison was between the site visit, plan and section representation and immersive virtual model.The evidences showed how the virtual experience provided a comprehension of the space more faithful to reality than the one given by the consultation of plans (Hermud, 2018).It remains difficult to estimate how much better is this way of communication due to numerous objective and subjective variables of the topic, but the quality and performances of this technology is undoubtedly going to rise in the next years.On its side, eye-tracking technology is widely used for various research purposes.Particularly interesting for the purpose are the studies about the orientation and wayfinding analysis in simulated environments.The study reproduced various types of buildings, with different test methodologies, all aiming to study human orientation.To evaluate the effectiveness of each system, showing that the orientation performances in the second case were higher (Vilar, 2012) the generic building reproduced had vertical and horizontal signage.A museum virtually reproduced under various circumstances, among which there was a fire emergency, it showed how this extreme circumstance affected the performances of the candidates, despite knowing that it was just a simulation (Cao, 2019).Vienna's central station reproduced for an immersive experience, involving elder people with the aim of investigating how their orientation changes between the virtual and the real station.The study showed strong and weak points about using this technology for simulation purposes and undoubtedly it was "important to investigate how effectively the user behavior and experience can be predicted based on a virtual reality experience" (Egger, 2016).The present research opens then, as a future developing, to the integration of neuronal analysis instruments, as the EEG helmet, previously used only in hospitals (Mavros et al., 2016;Roe et al., 2013).These devices are applied to study real contests (Aspinall et al., 2015;Mavros et al., 2012) are combined to virtual reality visors (Duchowski, 2017;Kuliga et al., 2015;Wilson and Soranzo, 2015).The integration of the VR allows controlling environmental factors in order to have confrontable experimentations (Bielik et al., 2015).
Figure 2: Actual atrium, its virtual reproduction and eyetracking analysis.

Three aspects of users orientation
The research is contextualises in wayfinding studies, renewed by digital tools and aims to understand how the architectural design affects in the behaviour, according to perception, understanding, and usage of space by the occupants.The first aspect concern the images elaborated by the brain through the multitude of inputs received from the environment, translating the theoretical work of Kevin Lynch (Lynch, 1960) about the perception of the cities into a smaller architectural scale.Despite the formal aspects are the first features in the project development, not always the theme of perception of spaces relates to the way they work in terms of distribution and connections between them.Sometimes it is due to a deliberate design option, other times is the result of negligence in the developing phase of the project.In both the cases, the result on the user is the same and it can be called amazement or daze according to his voluntariness.The second aspect is about the usage of the sorted perceptive information to obtain a mental map of the environment, a kind of an occupant's user manual of space.While the perception is an involuntary experience that affects all those who have an eye contact with space, the effort to interpret it involves only those who are interested in an interaction with it.This is a key phase for a correct use of architecture and when is not induced by the space itself, auxiliary methods are adopted, like signposting, involving certainly extra costs and occasionally bewilderment of the occupants if not efficiently realized.The third, the usage, is the most effective aspect and allow us to estimate the adequacy of a space to its purpose.Well known is the importance of wayfinding systems in crowded spaces like airports or hospitals; less evident are the consequences of those systems, if unsatisfactory, until they manifest themselves explicitly.The consequences of an inadequate emergency path, for example, can be terrible, even on a smaller scale than the two cited cases.Unfortunately, no exact rules governs one of those three factors, as the subjectivity of the individual strongly influences them.All the knowledge and regulations available are the result and combination of previous experiences and common sense.Apparently, the only ways to progress in the subject are to increase our experience, refine the common sense and rely on new technologies as a support to the purpose.

A method for new study perspectives
In this context, an empirical approach founded on multiple tests helps the study of the theme and supports the construction of interpretive models of the occupant's behaviour inside the designed space.The study of those three factors can be renewed using new digital tools: the simulation of immersive reality guarantees a new approach to the space, focused on the visualization topic.The study of perceptive polarities derived by eye tracking tools supports the analysis of space's influence, and designers may use the data collected by testing multiple candidates, to choose the more efficient solution in terms of clarity and legibility.This research introduces contemporary technologies in the topic, which purposes to provide new affordable experiences as a support during decision-taking phases of a design process and to assist and integrate the common sense of the designers.

The case study
The case study is an office building inside the campus of Tsinghua University in Beijing (China).In the building was detected a difficulty in user's orientation and a general ambiguity of the interior spaces.In particular, the repetitiveness of the interiors combined with improvable floor layouts were the deficiencies observed during various site visits, which made the building suitable for the purposes of the research.

Software and instrumentation
The complex architecture was surveyed and modelled in detail to be visualized in a virtual reality experience (Figure 2).In a first phase, the modelling program in use was Rhinocheros V5, while for texturing and UV mapping the program used was 3DS Max 2018.The textured model of the case study exported as .fbxinto the game development software Unreal Engine 4, was then setup for the immersive virtual experience.Author's confidence and experience choose the modelling and texturing software, while UE4 was chosen as development platform due to the availability of documentation and support, and to its node-based interface that allows to code complex functions without a specific digital knowledge.As device for the visualization of the virtual experience was used an HTC Vive powered with a workstation with the following specifics: -processor: Intel I5 3.40 GHz -graphic: Nvidia Ge-Force GTX 1060, 4 Gb -RAM: 16 Gb The researches preferred Head Mounted Display (HMD) among others, because of its greater nominal tracking area.A specific infrared plugin lenses specifically designed for the HTC Vive by 7invensun was used to fulfil the eye-tracking function.These lenses, called A-Glass, are insert inside the Vive between the user's eyes and the screens, without compromising the virtual reality experience or its comfort.It is also possible to insert graduated lenses for eventual poor eyesight.A USB external port of the Vive connects the hardware connection between lenses and HMD, while the use of A-Glass output data inside UE4 is possible by using a plugin software provided by 7invensun.

The test
A group of eight candidates was involved to play the virtual experience in which they had to succeed in a series of wayfinding and orientation tasks inside the virtual environment, none of them knew the modelled building.
The first part of the test consisted in a calibration phase, during which the candidate had to fix determined points in the virtual space in order to calculate the variables of the algorithm that transform the output of the lenses, a twodimensional vector of normalized screen coordinates, in a precise point of the 3D space.When the calibration phase was completed, the player was teleported at the entrance of the virtual building, ready to start the testing phase.The candidate wearing the HMD will hear five targets, called out one by one, all consisting in a list of five locations inside the architecture.The order was the same for every test and the last target was always the closest emergency exit.For every target, there were two sub-tasks: the path-finding task consisted in reaching independently that specific room inside the building.Once completed this step, the following one was the orientation task and it consisted in pointing the head towards the previous room-objective, trying to intercept its volume in the 3D space with a straight line traced just once from the player's position according to the orientation of the HMD (Figure 4).The aim of the path-finding task was to analyse the paths inside the building, to understand where the candidates went, where they got lost, whether they got lost.The algorithm recorded the player's movements, saving his positions in an x,y,z vector calculated with a frequency of 5 Hz and listed in a .txtfile available at the end of the test for each candidate.Afterwards, the position vectors translated in line-type graphs in the spreadsheet program Excel, overlapped the floor plans of the building.The orientation task aimed to evaluate the orientation of the candidate inside the architecture and the awareness of the path just completed.The results of the second part of the test recorded as Boolean data type in a row inside the cited .txtfile, on one part, the modelled building had no wayfinding signposting, while on the other part, it appears as actually present in the real architecture.This latter distinction evaluates the quality of the actual wayfinding systems and shows how they affect the orientation, when they do it.Reassuming: eight candidates for eight tests, only 50% with signposting.Five targets each test, two tasks for each target: path finding and orientation (figure 5).During the whole duration of the tests, the eye-tracking lenses analysed the eyes movements and the gaze point of each candidate, recording them in a series of detection planes positioned in proximity of walls, floor and points of interest in general, with a frequency of 30 Hz.Those planes were invisible to the player during the test but they changed their colour, according to a heat map gradient, if the player's gaze fell on them.This way of visualization of the eye-tracking data showed up where the candidates were looking, directly in the 3D space.The tests with signposting allowed evaluating the efficiency of wayfinding systems, whether they were consulted or not.In the test without signposting, it was helpful to understand where the candidates were expecting to find information and which portion of the space attracted their attention.
Figure 6: Screenshot from one of the tests.

Results
The results of orientation tests, plus data about their movements and eye tracking were recorded and stored in a comparable way; obtaining a behaviour database of those candidates for that specific building.The analysis of the movements in the building and the results of the orientation tests made it possible to locate where the candidates got lost.As well, the eye tracking data gave us information about the points of interest of the spaces according to the player's individuality.The intersection of these data highlighted the weaknesses of the building in terms of legibility of the spaces and efficiency of the wayfinding systems.Particularly, the poor success ratio of the orientation tasks shows how the candidates could not orientate themselves inside the building in most of the cases, even though it has slightly increased in the tests with the presence of signposting (Figure 13).The paths recordings confirmed the bewilderment caused by the monotony of spaces, with the same slight improvement seen before with the present of signposting.For the sake of brevity here are shown the recordings of the most relevant example: the last target, where was asked to find the emergency exit, shows a countertrend in which the candidates got wrong mostly in presence of wayfinding signals, due to their incorrect placement (Figure 7).The eye-tracking data took from one of the tests confirms that: the emergency signposting that incorrectly brings to the main entrance is well notable while the corridor of the safer floor exit goes totally unnoticed (Figure 8-9).

Refurbishment proposal
The study crosschecked paths data, orientation tests and eyetracking heat maps in order to highlight the improvable aspects of the actual architecture.Considering the detected deficiencies of the building, a proposal of renovation was modelled and transformed in a virtual reality experience in the as well as for the real building.The main feature of the proposed refurbishment is a full height atrium that breaks the rigid by-layers perception of the building, augmenting the visual transparency of the interiors and helping the understanding of the space (Figure 10).The central new space separates different working areas that have been further distinct by different colours and number codes.Floor layouts as well were changed for a more ordered distribution and less ambiguous spaces.The exteriors and aesthetics are omitted in this article.

Results comparison
Using the same algorithm in the new-modelled building, the tests were repeated with the same candidates .The comparison between the behaviour databases recorded in the two virtual buildings was the evaluation method to evaluate the quality of the proposal.As done before, here are the relevant results.The orientation tests show a success ratio doubled compared to the original building, while is reduced the difference between the tests with signposting and those without it (Figure 13).This result can be interpreted as a better understanding of the refurbished spaces from the candidates and a lower weight of signposting in the orientation process, due to a better legibility of the interiors.The paths recordings of the last target show how this time the emergency exit has been found easily with or without the wayfinding systems, with only a single occasion of misunderstanding when there were no signals (Figure 14).An example of eye-tracking heat maps, took in the floor where was asked to find the emergency exit, highlights how the emergency door was noticed regardless the presence of indications.The same example shows how the signposting was widely consulted when present, and therefore well positioned .The comparison of motion tracking and the orientation tests showed a sharp decline in bewilderment situations, while the eye tracking data confirmed the efficiency of the new wayfinding system proposed.According to the compared data is then possible to affirm that the candidates were more confident inside the proposed building and therefore that the architecture gained in clarity and legibility thanks to the renovation.

CONCLUSIONS
The research does not focus on the case study itself, but on the approach to the simulation topic, aimed to define a methodology replicable in different conditions and cases.The innovative approach of the research is in the application of the eye-tracking methodology to determinate spatial references: the immersive reality guarantees the possibility to project data directly into the 3D environment on defined geometries, bypassing the usual adoption of screens to remedy the absence of biunivocity in the individuation of point of interests in the usual eye tracking applications.Another relevant feature of this approach to the wayfinding problem is its immateriality: as everything happens in virtual environment, the investigation involves a theoretically infinite number of configurations at the cost of the sole modelling.In addition, the size and transportability of the required equipment allows reaching a wider number of cavies even outside the usual laboratory environment.Introducing scientific data and processes, the path also helps to identify the improvable aspects and proposes specific enhancements to increase the scientific relevance of the method itself.

Figure 3 :
Figure 3: The HMD Vive components with the eye-tracking lenses.

Figure 4 :
Figure 4: Graphic explanation of the orientation task.

Figure 5 :
Figure 5: Overview of the developed methodology.

Figure 7 :
Figure 7: Path-finding for the Target 5 (emergency exit) in the actual building, left without signposting, right with signposting.

Figure 10 :
Figure 10: Concept and sections of the building before (left) and after (right) the refurbishment proposal.

Figure 11
Figure 11-12: The entrance of the refurbished building and its eye-tracking analysis from one test.

Figure 14 :
Figure 14: Path-finding for the Target 5 (emergency exit) in the refurnished building, left without signposting, right with signposting.

Figure 15
Figure 15-16: Eye-tracking data in the refurbished building of two tests, on the top the test without signals, down a test with signals.