THE SELIMIYE MOSQUE OF EDIRNE, TURKEY – AN IMMERSIVE AND INTERACTIVE VIRTUAL REALITY EXPERIENCE USING HTC VIVE

: Recent advances in contemporary Virtual Reality (VR) technologies are going to have a significant impact on veryday life. Through VR it is possible to virtually explore a computer-generated environment as a different reality, and to immerse oneself into the past or in a virtual museum without leaving the current real-life situation. For such the ultimate VR experience, the user should only see the virtual world. Currently, the user must wear a VR headset which fits around the head and over the eyes to visually separate themselves from the physical world. Via the headset images are fed to the eyes through two small lenses. Cultural heritage monuments are ideally suited both for thorough multi-dimensional geometric documentation and for realistic interactive visualisation in immersive VR applications. Additionally, the game industry offers tools for interactive visualisation of objects to motivate users to virtually visit objects and places. In this paper the generation of a virtual 3D model of the Selimiye mosque in the city of Edirne, Turkey and its processing for data integration into the game engine Unity is presented. The project has been carried out as a co-operation between B İ MTA Ş , a company of the Greater Municipality of Istanbul, Turkey and the Photogrammetry & Laser Scanning Lab of the HafenCity University Hamburg, Germany to demonstrate an immersive and interactive visualisation using the new VR system HTC Vive. The workflow from data acquisition to VR visualisation, including the necessary programming for navigation, is described. Furthermore, the possible use (including simultaneous multiple users environments) of such a VR visualisation for a CH monument is discussed in this contribution.


INTRODUCTION
Virtual Reality (VR) will change our future life.Although VR is not very new, it is currently at the beginning of a technological transition.The term Virtual Reality was introduced the first time by the author Damien Broderick in his science fiction novel "The Judas Mandala" published in 1982.As early as 1962, Morton Heilig built the Sensorama, a machine that is one of the earliest known examples of immersive, multisensory (now known as multimodal) technology and which could be named as the first VR system (Rheingold 1991).Nevertheless, it took another thirty years until the first Head-Mounted Display (HMD) for the mass market was released with the VFX 1 from Forte (Cochrane 1994).Unfortunately, this headset caused nausea and disturbances of equilibrium for users due to the long latency when visually updating the display.After almost another twenty years the new HMD Oculus Rift was announced by Palmer Luckey, which really worked and which rang in the new era (Desai et al. 2014).
It is already obvious that this new technology will offer great opportunities for many applications such as medicine, engineering, computer sciences, architecture, cultural heritage and virtual restoration.VR typically refers to computer technologies that use software to generate the realistic images, sounds and interactions that replicate a real environment, and simulate a user's physical presence in this environment.Furthermore, VR has been defined as a realistic and immersive Figure 1.The mosque of Edirne, Turkey -exterior view (left), detail view (centre) and interior view (right) The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-5/W1, 2017 simulation of a three-dimensional environment, created using interactive software and hardware, and experienced or controlled by movement of the user´s body or as an immersive, interactive experience generated by a computer.VR offers an attractive opportunity to visit objects in the past (Gaitatzes et al. 2001) or places, which are not easily accessible, often from positions which are not possible in real life.Moreover, these fundamental options are increasingly being implemented today through so-called "serious games", which embed information in a virtual world and create an entertaining experience (edutainment) through the flow of and interaction with the game (Anderson et al. 2010;Mortara et al. 2014).The first virtual museum using the VR system HTC Vive as a HMD for immersive experiences was introduced by Kersten et al. (2017).
BİMTAŞ, a company of the Greater Municipality of Istanbul, Turkey, generated a virtual 3D model of the mosque of Selimiye for the Edirne City Government to promote the monument for tourism purposes.This 3D model has also been processed in co-operation with the Photogrammetry & Laser Scanning Lab of the HafenCity University Hamburg, Germany, for VR application to become the first historical monument in Turkey that is visualised as an immersive and interactive experience using the new VR system HTC Vive.

THE SELIMIYE MOSQUE IN EDIRNE, TURKEY
The Selimiye Mosque (Fig. 1) is an Ottoman imperial mosque, which is located in the city of Edirne, Turkey.The four pencilshaped minarets, which are the tallest ever built (71 m from the ground to top), give the mosque an unforgettable profile.A detailed description about the mosque is given by Blair & Bloom (1995).The mosque was commissioned by Sultan Selim II, and was built by the architect Mimar Sinan between 1569 and 1574 as a masterpiece and as one of the highest achievements of Islamic architecture.This grand mosque stands at the centre of a külliye (complex of a hospital, school, library and/or baths around a mosque) that comprises a medrese (Islamic academy teaches both Islamic and scientific lessons), a dar-ül hadis (Al-Hadith school), a timekeeper's room and an arasta (row of shops).In this mosque Sinan employed an octagonal supporting system that is created through eight pillars incised in a square shell of walls.The four semi domes at the corners of the square behind the arches that spring from the pillars, are intermediary sections between the huge encompassing dome (31.25 m diameter with spherical profile) and the walls.
Surrounded by the four tall minarets, the Mosque of Selim II has a grand dome atop it.Around the rest of the mosque were many additions: libraries, schools, hospices, baths, soup kitchens for the poor, markets, hospitals, and a cemetery.These annexes were axially aligned and grouped if possible.In front of the mosque sits a rectangular court with an area equal to that of the mosque.The innovation however, comes not in the size of the building, but from the organization of its interior.The mihrab is pushed back into an apse-like alcove with a space with enough depth to allow for window illumination from three sides.This has the effect of making the tile panels of its lower walls sparkle with natural light.The amalgamation of the main hall forms a fused octagon with the dome-covered square.Formed by eight massive dome supports, the octagon is pierced by four half dome covered corners of the square.The beauty resulting from the conformity of geometric shapes engulfed in each other was the culmination of Sinan's lifelong search for a unified interior space.The mosque, together with its külliye, was included on UNESCO's World Heritage List in 2011.

WORKFLOW
The following workflow has been carried out to generate a virtual 3D model of the Edirne mosque: (1) data acquisition using terrestrial laser scanning with the Riegl VZ-400 scanner, (2) 3D modelling using point clouds for generating 3D meshes with Autodesk ReCap or Geomagic studio and for solid modelling with 3ds Max, (3) texture mapping of polygon models, (4) generation of videos of the textured 3D model, (5) data reduction (polygon decimation) for VR application, (6) data conversion for the import into the game engine Unity, (7) implementation into the HTC Vive including the programming for the motion control, and (8) immersive and interactive visualisation of the mosque in the VR system HTC Vive.

DATA ACQUISITION AND 3D MODELLING
The scanning of the mosque and its environment took almost one month (some days in September and November 2012) using one terrestrial laser scanning system Riegl VZ-400 and three staff members of BİMTAŞ (Fig. 2).In total, 110 interior scans and 447 exterior scans of the mosque were acquired with an average scan density of six points per cm 2 .Reflective targets were only used at some scan stations for geo-referencing of the point cloud.For later colorizing of the laser scanning point clouds (Fig. 3) an integrated Nikon D700 camera with 20 mm lens was placed on top of the Riegl scanner.Five images per 360 o view per scan were acquired.All scans were registered in the Riegl RiScanPro software with a precision of approximately 2-3 mm.In total a data volume of 3.5 billion laser points were scanned.Processing in RiScanPro took approximately eighteen person weeks.To geo-reference the laser scanning point clouds for the interior and exterior areas of the mosque and to integrate the data into the national coordinate system a 3D geodetic network, including the reflective targets, was measured prior to scanning using total stations and GNSS.For later texture mapping of the solid 3D model of the mosque (Fig. 5) photographs were taken using the Nikon D90 cameras during the field campaign.These images were radiometrically enhanced in Photoshop before texture mapping.

THE GAME ENGINE UNITY
For the virtual reality application the game engine Unity was used.A game engine is a software framework designed for the creation and development of video games for consoles, mobile devices and personal computers.The core functionality typically provided by a game engine includes a rendering engine for 2D or 3D graphics to display textured 3D models (spatial data), a physics engine or collision detection (and collision response) for the interaction of objects, an audio system to emit sound, scripting, animation, artificial intelligence, networking, streaming, memory management, threading, localisation support, scene graph, and may include video support for cinematics.A game engine offers the ability to control the course of the game and is responsible for the visual appearance of the game rules.For the development of a virtual reality application, game engines offer many necessary concepts with much functionality so that users can interact with the virtual object.The solid 3D model of the mosque was transferred to Unity using FBX file format.However, the data volume of the model had to be reduced from 6 million polygons to fewer than two million polygons to guarantee sufficient computer performance during VR visualisation.At the end of processing in Unity, an executable program for the VR system HTC Vive was developed.

THE VR SYSTEM HTC VIVE
HTC Vive (www.vive.com) is a virtual reality headset (Fig. 7) for room-scale virtual reality.It was developed by HTC and Valve Corporation, was released on 5 April 2016, and is currently available on the market for EUR 899.Basic components are the headset for the immersive experience, two controllers for user interactions and two "Lighthouse" base stations for tracking the user's movement (Fig. 8).The visit offers the possibility of experiencing the mosque from a real person point of view and interaction scheme.For this purpose, the controlling positions have been defined by the possibility of free movement by the user.To bridge long distances in the virtual object, teleportation or fly-through functions can be developed for navigation of the user.The device uses a gyroscope, accelerometer, and laser position sensor to track the head's movements as precisely as one-tenth of a degree.Wireless controllers in each hand, with precise SteamVR-tracking, enable the user to freely explore virtual objects, people and environments, and to interact with them.
The VIVE-controller is specifically designed for VR with intuitive control and realistic haptic feedback.The Lighthouse system uses simple photo sensors on any object that needs to be captured.To avoid occlusion problems and increase tracking accuracy, this is carried out with two lighthouse stations that sweep structured light lasers within the space.Fig. 9 shows the VR system HTC Vive in action illustrating the view in the HMD on a TV screen.

DATA REDUCTION FOR VR APPLICATION
The final CAD model of the Selimiye mosque comprised 6.5 million polygons.This amount of data was too large for an acceptable performance of the VR application in the game engine Unity.Thus, the number of polygons had to be significantly reduced to ensure smooth performance during the motion in and interaction with the virtual 3D model using the VR system HTC Vive.In the first step, the number of polygons was reduced from 6.500.000 to 1.508.135.However, this polygon reduction caused a number of problems in the solid 3D model, which were visible during the visual inspection of the virtual model in the VR system HTC Vive.The problems that occurred are summarized in the following:

THE VR IMPLEMENTATION
For a virtual walk within a real environment, the dimensions of the object must be to scale.The basis for the VR visualisation was therefore the textured CAD model generated in 3ds Max.
After data import into the Unity engine some material parameters had to be adjusted because even after many years of game design the transformation of 3D models into game engines is still a task to be solved completely.Locomotion in VR is currently also an area of active research.McCaffrey (2017) gives an overview of various locomotion methods and their implementation in the game engine Unreal.Most experiences implemented on the hardware, described in section 7, use natural locomotion where real walking movements are translated to virtual movements and the limitation of the tracked physical space is extended by teleportation.Due to the immense height of the mosque's interior and exterior, an artificial locomotion scheme was utilized that is not limited to the ground plane.By pointing the motion controllers, the direction of a flightpath is selected and the user is continuously moved while pressing a trigger style controller button.The velocity of the movement can be adjusted using the Vive touchpad where the information is also displayed in form of a small UI (Fig 12).The movement feature was implemented using the SteamVR package from the Unity asset store.
Virtual reality utilizing a HMD is a solitary task by definition, because all outside senses are dampened to enhance the immersion and, therefore, the experience.To create a more social experience, multiple users can join the experience to explore the virtual object together as avatars while each user is at a different real world location using a HMD (Fig 13).The independent networking solution Photon (Photon Unity Networking) is used to implement the synchronisation of multiuser movements.Voice is acquired through the included microphone in the HMD and played back for all users with respect to the speaker's position (spatialized audio).Photon also provides the server infrastructure and is free to use for a moderate user count.This VR system provides a truly immersive experience that allows the users to step inside a virtual building environment, such as the mosque, without having ever seen the building in real life.The multi-user functionality offers new capabilities for interested parties to hold interactive discussions about aspects such as the architecture, structural analysis, building history and virtual restoration of a monument or a building.Moreover, the VR application enables geometric quality checking of the modelled 3D data during the VR visualisation.
However, comprehensive and time-consuming data recording, modelling and texture mapping of the real environment is an essential requirement for the creation of an immersive visualisation in virtual reality.In addition to this, a smooth visualisation can only be guaranteed if the data volume, i.e. the number of polygons and texture resolution, of the environment to be visualised has been reduced and adapted respective to the available computer performance.It does not matter whether the monument or the building is a reconstruction of something previously destroyed and forgotten, such as the castle of Bad Segeberg, Germany (Deggim et al. 2017), or if the object is a surviving monument, such as the imperial cathedral of Königslutter, Germany (Kersten et al. 2012)  Furthermore, the emerging technology Augmented Reality also offers great potential by combining the advantages of a virtual with a real visit to a monument.It also enables an in situ monument visit using a Smartphone or tablet for digital superimposition of a historic building state onto the current reality, for example at sites of (former) historic ground (Canciani et al. 2016).

Figure 2 .
Figure 2. Data acquisition with the terrestrial laser scanning system Riegl VZ-400 on top of a minaret (left) and inside the mosque (centre and right)

Figure 3 .
Figure 3. Part of the coloured point cloud of the Edirne mosque -overview (top) and detail (bottom)The 3D modelling of the mosque took 24 person weeks to perform.In this step, geo-referenced point clouds were imported to Autodesk Recap and Geomagic studio (for smaller details) for the generation of 3D meshes (Fig.4).These meshes were used as the basis for solid modelling in the program 3ds Max to derive CAD drawings and the 3D model.Similar workflows were presented byDursun et al. (2008) andKersten et al. (2009).

Figure 4 .
Figure 4. Mesh used for solid modelling in 3ds Max (top left) and 3D CAD drawing of the front door (top right) and section of the interior area (bottom) of Edirne Selimiye mosque

Figure 5 .
Figure 5. Textured 3D model superimposed with the wireframe view

Figure 6 .
Figure 6.Still video frames of the video sequence of the Selimiye mosque of Edirne -perspective views of the exterior (top and centre) and interior (bottom)

Fig. 9 .
Fig. 9.The Virtual Reality System HTC Vive in action using the controllers for interacting in the environment (a) gaps, texture and UVW shifts on the objects, (b) double faces on the objects, (c) missing elements which are available in the original data structure of the mosque, and (d) errors with proportionality of objects in comparison with the original structure of the mosque.However, these above-listed problems and errors were corrected by additional manual modelling and reconstruction.Furthermore, all dense polygon objects were replaced by low polygon objects for further data reduction.Finally, the number of polygons was reduced to 901.133.The mosque objects with reduced polygons are indicated in colour in Fig. 10.Fig 11 illustrates two examples of polygon reduction using two different objects.

Fig. 10 .
Fig. 10.Coloured object parts of the mosque indicate all objects with polygons reduced

Fig. 12 .
Fig. 12.The Virtual Reality System HTC Vive in action using navigation controller for the motion in the environmentThe multi-user functionality was tested between Hamburg and Istanbul on February 14 th 2017, when four users simultaneously visited the virtual model of the mosque to discuss and to check the geometric quality of the modelled 3D data during the VR visualisation.During the joint virtual visit, the users communicated with each other about the content via microphone.At the groups of users were based at two different locations, Hamburg and Istanbul, travel expenses could be avoided.

Fig. 13 .
Fig. 13.Multi-user feature of the Virtual Reality System HTC Vive presenting two avatars with controllers10.CONCLUSION AND OUTLOOKIn this contribution, the digital workflow from data acquisition to the immersive visualisation of the Selimiye mosque of Edirne has been presented.Based on the modelled and textured 3D data in the game engine, an immersive virtual reality visit was developed utilizing the new Virtual Reality System HTC Vive.This VR system provides a truly immersive experience that allows the users to step inside a virtual building environment, such as the mosque, without having ever seen the building in real life.The multi-user functionality offers new capabilities for interested parties to hold interactive discussions about aspects such as the architecture, structural analysis, building history and virtual restoration of a monument or a building.Moreover, the or many others.Cultural heritage monuments are ideal objects for immersive virtual reality visualisation for applications in architecture, building information modelling and restoration.Virtual reality is an entirely new medium whose true purpose is slowly being realized.CNET (2017) describes everything the users need to know about what VR is and how it will affect our life in the near future by the following applications: entertainment, artistry & design, gaming, education & simulation (e.g. in medicine, chemistry, physics, astronomy), tourism & exploration, psychology & meditation, real estate & shopping, and social & telepresence.