DEVELOPING AN EDUCATIONAL CULTURAL HERITAGE 3D PUZZLE IN A VIRTUAL REALITY ENVIRONMENT

The ways our cultural heritage reserve is preserved and disseminated to the public have changed significantly, with the use of immersive technologies, such as virtual reality environments, and serious games. Nowadays, these technologies are also exploited for developing interactive informative applications, to support historical education and enhance museum visits, physical or virtual, especially to younger generations. The field of edutainment, educational entertainment, has been rapidly developing during the last 10 or 15 years. The main goal of this research is to develop an educational 3D puzzle-like serious game which can operate within a virtual reality environment while aiming towards the dissemination of cultural heritage content to the younger public, i.e., students, children etc., through a pleasant gamification process. The cultural heritage objects used are an ancient Greek temple and a statue of the Roman era, whose high-resolution fully textured 3D models were available from previous projects. The game application was developed in Unity game engine with suitable coding to enable the smooth execution of the 3D puzzle solution. The application verified that it is more interesting to learn about cultural assets through a game than in the conventional ways, and even more when it is implemented within a Virtual Reality environment, where the contact with the assets appears to be more direct and realistic. The same application can also be utilized in different educational areas and can be expanded by the inclusion of other digital assets. * Corresponding author


INTRODUCTION
Preserving cultural heritage and disseminating it to the public has been implemented using digital technologies such as Virtual Reality and Augmented Reality combined with Serious Games during the last decade. Nowadays, these immersive digital technologies are utilized for storytelling and educational purposes, offering a different approach to obtaining knowledge. Extended Reality is also exploited for interactive lessons in a direct environment, or enhancing museum visits by creating life-like models of destroyed artifacts or offering opportunities for remote visits.
Cultural Heritage, according to UNESCO Institute for Statistics (2009), includes artifacts, monuments, sites and museums that have a diversity of values and are of great anthropological and scientific significance. It can be tangible, intangible and natural. Serious Games are evolving applications that do not have only entertainment as their vital purpose, but education as well as reported by Michael & Chen (2006), known for fusing various scientific domains with gaming. In other words, Serious Games are computer applications that aim at combining aspects of tutoring, teaching, training, communications and information with the recreational element of video games according to Alvarez & Michaud (2008). They are extensively used in academic surroundings, since it is proven that they enable knowledge transfer, motivation and skills training. Extended Reality is the umbrella term that describes all environments that combine reality and virtuality, with Virtual, Augmented and Mixed Reality being the most popular ones (Figure 1). The main difference between VR and AR is that the latter amplifies the physical world using digital assets, whereas the former digitally recreates realistic simulations of the physical world that the users interpret as existent and interact with them in real time. The most important characteristic of VR, and also the measure of success of a VR environment, is immersion, which translates as the sensation the users get of their existence in a virtual environment. Other important parameters are the level of interaction and the information intensity provided by the VR system. Mixed Reality refers to the merging of both real and physical worlds to create new visualizations and is described as a continuum that extends from the completely real to the fully digital environment. The main purpose of this research is to show how starting from any 3D model, it is possible to develop a didactic game tool in order to convey the history and the qualities of cultural heritage assets in an engaging and intuitive way. More specifically, it is focused on developing an educational 3D puzzle that operates within a Virtual Reality environment, supporting assets of cultural heritage whose 3D textured models have been created using photogrammetric or similar methods. Its goal is to aid users to learn more about the cultural heritage assets in question through a gamification process. This paper describes the related work that was the main influence for the research and the framework in which the application was developed.

RELATED WORK
Interactive digital storytelling and edification on cultural heritage is expanding to the point that is now considered a common practice. Already in 2007, the concept of Serious Games was examined for the addition of pedagogy that differentiates them from conventional computer games, and their influence on management and leadership challenges as well as reported by Susi et al (2007). In 2010, although already more than a decade ago, very interesting research was published by Anderson, et al. (2010) about the effort being made on reconstructing archaeological sites in digital form for educational purposes. An example from this research is "Roma Nova", which was the largest digitisation project at the time, aiming at the production of a version of Rome at 320 AD, providing an insight into the social life, architecture and design of the city.
The number of Extended Reality applications with academic backgrounds as their main goal is also increasing. Recently, as part of a research by Okanovic, et al. (2022), various VR and AR applications were developed, with the aspiration of sharing the story of Bosnia and Herzegovina by giving users the active role and ability to influence the flow and content of the story, as well as the nature of knowledge they will get. As it is mentioned in this publication, the success of such applications is measured by the user experience, and mainly edutainment, i.e. how well education and entertainment are combined. It is underlined that it's very important to make Cultural Heritage Serious Games interesting and approachable to the regular visitor, delivering the proper amount of information in the best way possible, since "stories are for people, not things". A handful of similar presentations have also been developed by the Laboratory of Photogrammetry of National Technical University of Athens, e.g., the attempt of creating a serious game in the Ancient Agora of Athens, in which the player could gain knowledge about the ruins by answering relevant questions published by Kontogianni & Georgopoulos (2015) and the creation of a Serious Game located in the Stoa of Attalos in Ancient Agora of Athens which aims at raising people's awareness of the Conservation Theory and Practice, reported by Koutsaftis & Georgopoulos (2015). Also, research has been done by the aforementioned Laboratory on the influence of Serious Games on Cultural Heritage and Tourism, by  and last but not least, on the procedure of developing 3D textured models that will be used in Serious Game applications as reportd by .
VR applications have a major impact on virtual museums, which generally aim at the elimination of the obstacle of physical space and offer a chance to remote visitors. A great example by Kiourt et al. (2016) is the development of DynaMus, a fully dynamic 3D virtual museum framework based on Serious Games that presents an innovative way to easily and freely build a virtual environment with digital content already available on the Web.
Of course, Serious Games in virtual environments are not restricted to the cultural heritage domain. Many VR applications have been developed with the goal of assisting users to overcome health issues and psychological traumas, for example PTSD or the fear of heights, or train on specific skills. Lately they are used at a great length in architecture and urban design, offering a deeper understanding of buildings and space and improving the design process. It is worth mentioning that the idea of a 3D puzzle in a VR environment, which is the theme of this research, generated from an application that pointed towards learning human anatomy, targeting students who trained to become doctors as reported by Pohlandt, Preim, & Saalfeld (2019). In that application, users assemble the human skeleton from scratch, gaining knowledge not only on the human anatomy as a whole but on each individual part as well. The research emphasizes on the four steps of learning: preparation, presentation, practice and performance, and cultivates every one of them throughout the process. This was a significant application is great, since it was accepted as a learning technique, smoothing the way for similar products to be put into operation.

METHODOLOGY
The present application was developed by following a pipeline consisting of 4 stages. First of all, the necessary data were collected and modified. Secondly, an experimental asset was used to develop a simple 3D computer game, in order to test the use of the Game Engine and the necessary coding, followed by the third step, the insertion of the modified data in the computer game. Finally, the game is converted into a serious game VR application in the form of True and False questions based on the assets.

3D models used and necessary modifications
The assets that were selected are the 3D models of the Ancient Temple of Demeter ( Figure 3) located in Naxos Island and the sculpture "Emperor and Barbarian" (Figure 4), which was found in the Ancient Agora of Athens. The aforementioned objects were selected firstly because of their detailed surface that made the assembly of a puzzle interesting and secondly because they have an impressive history known to very few people. More specifically, the Ancient Temple of Demeter was discovered in 1949 buried under a church. It is constructed with white marble and is considered a variant of the Ionic order and also a precursor of Parthenon of Athens, as it dates almost a century before it. It has a unique architecture since it is square instead of elongated and is oriented in the axes of North -South instead of the common East -West. The temple is devoted to Demeter, goddess of agriculture, her brother Apollon, the god of music, and her daughter Persephone, who according to Greek mythology, was abducted by Hades and cursed to live in his realm for six months every year. The sculpture, on the other hand, was found in 1972 by the American School of Classical Studies which conducted excavations at the time. Only the legs of the emperor and part of his cloak and the biggest part of the barbarian are preserved, the rest of the sculpture is scattered and cannot be attached to the main body. The emperor displays either Trajan or Adrian, both of whom were great Roman Emperors and led the Roman Empire to rise socially and architecturally. The 3D models of the assets were created in previous projects by Giannakoula (2018) and Ravani (2018) using photogrammetric techniques. In order for the assets to be used in the game, they had to be simplified because they were of high resolution. More specifically, the Temple had to be decimated because the number of surface triangles was so large that the object was unmanagable by the game engine. Moreover, the original 3D model was missing some information because of occlusions and vegetation, so the hollow parts had to be filled. The structure required only modification in data size, so the amount of surface triangles was reduced and they were converted to quads without losing any detail. All these necessary actions were completed in open source software.

Application Development
The application is developed in Unity 3D Game Engine via C# programming in Visual Studio. In order to shape the game as a puzzle, the experimental and the selected assets were virtually broken into pieces using Blender software. The concept of the game is to randomly distribute the pieces within a specified range in the scene and then reassemble them in their correct positions. In the final form of the game, the user has to pick one asset, randomize the positions of its pieces in the scene, answer correctly to one "True or False" question for each piece to be moved and finally move all of them in their initial positions.

Experimental stage:
The result of this stage is a 3D computer game that can be played using mouse and keyboard inputs and constitutes the base for the development of the whole game. The experimental asset is a cube (Figure 5), which is broken into 7 pieces. The user has to press the space bar to randomize the position and orientation of each piece and then move and rotate them using the left and right mouse buttons respectively. One of the pieces remains immovable, so it can be used as a reference point for the set up. The positions can be randomized only once to avoid any accidents in the gameplay, so the action of the space bar is disabled after the first hit. To make the translation along all three possible axes, since the computer mouse can only move in x and y, movement of the camera was added, enabling the user to wander around the object, zoom in and out and reposition each piece freely with a combination of all actions. Moreover, to offer a better view of the asset, the camera rotates around the cube when the game begins and stops with the press of the space bar. The interaction with the pieces is realized using a draganddrop mechanism coded in C# and based on the detection of a ray, which originates from the built-in camera with every mouse click and tracks the location of the mouse (Figure 6). Every time the ray lands on a piece, interaction with it is enabled. To make the detection of the pieces easier, they are all added in a specific layer and the ray is activated whenever it hits the content of it. When a piece approaches its initial position, it is automatically sent to it, provided it is at a minimum specified distance away from the origin (0,0,0), and rotated by a minimum specified angle from (0,0,0,0). For the cube, the threshold for the distance is set to 0.45 meters, since it is more difficult to reposition the 3D pieces only via mouse, whereas for the angle 25 degrees. As a last step, for every piece that is correctly placed, the part of the code that enables interaction is disabled to avoid moving it again by mistake.

Inserting the assets:
In the third stage, the two assets are inserted in the computer game, replacing the experimental cube. The assets are "broken" into pieces beforehand, again in Blender software. Specifically, the temple which is larger and has a more detailed surface, is broken into 10 shreds whereas the sculpture which is morphologically simpler, into 5. The shreds are then inserted in different scenes in the game engine, in order to create two levels of difficulty, adding all the necessary physical characteristics to make them interactable. The number of pieces every asset is broken into, can be defined by the user. The actions remain the same, with only some minor adjustments, for example the threshold values are different and the range value as well, so as no shred will appear off the scene. In this step, some sound effects are added when the pieces are scattered to make the game more fun.

Converting to a VR Serious Game:
In this stage, the game gets its final form. The equipment used in the Virtual Reality environment is the HTC Vive Pro system which consists of one HMD and two controllers (Figure 7), each of which has 5 different buttons: touchpad, trigger, two grip buttons and a menu. The transition to VR is made in Unity using the Steam VR extension and all the actions are coded in C#. The changes made are of great significance and they amplify the simplicity of VR applications both in developer and user terms. It is worth mentioning that the detection of whether the player triggers an action via the controllers is coded through Boolean variables.

varwin.com/en/latest/)
As a first step, an environment is designed for both scenes. The temple is placed on the ground, whereas the sculpture on a table. The philosophy of the game remains the same as the conventional computer application, however the means with which the goal is achieved have changed. All camera actions are removed, since the players themselves have the role of the camera in the VR environment, through the headset which tracks every move the players make in real life and translates it into in-game movements. It is proven, however, that moving inside VR environments often causes feelings of dizziness, so it is decided that apart from walking around the scene, the players can preferably teleport in specified areas in the scene ( Figure 8) and move closer or further from their point of view by pressing and scrolling their thumb on the right touchpad. Nevertheless, the pieces' positions can only be randomized once and one piece remains at a stationary position to be used as the reference point, but the drag-and-drop mechanism is no longer necessary, as there are built-in tools enabling interactions with specified objects in the scene, by determining particular layers. In detail, whenever the right grip button is pressed while the controller hovers over a piece, this can now be moved and rotated in every direction as if the players hold it by themselves. The threshold value which makes the piece move automatically to its correct position decreases to 0.1 meters, because it is easier to reposition objects in VR, and when it is there, interaction with it is disabled but instead of deactivating the code, the piece jumps to a layer which is removed from the list of layers with which the player can interact.
The conversion of the VR puzzle to a serious game application is made by adding "true" or "false" questions about the history and architecture of each asset (Figure 9). The questions are written on canvas objects. A random question appears every first time the player hovers their right hand over a wrongly placed piece with the purpose of moving it. Interactions are disabled until the player answers the given question correctly. In case where they fail to answer one question, a new one appears after 3 seconds with a text that asks the player to try again, and so on. The player selects either the true or false button with a laser pointer that springs from the left controller. Each question appears only once, and there are enough questions to ensure that all the pieces can be moved. More specifically, there are ten questions for the sculpture and twenty for the temple. The game is over when all the pieces are in their initial position, and this is detected by counting the contents of the new layer to which they jump.

Additional details:
When the game loads, a menu appears in which the players are asked to select one of the assets in order to proceed (Figure 10 10). The selection is categorized to "easy" or "difficult", according to the number of pieces. The sculpture, cut into 5 pieces, represents the first category, while the temple, cut into 10 pieces, the second one.

Figure 10: Selection Menu
To make the questions easier, when an asset is selected, instead of transferring the player directly to the scene, a text appears containing information about the object, enabling the players to read and learn about it before proceeding to the game ( Figure  11). The questions are strictly sourcing from these texts. In addition, when the game is complete, the players are transferred to a new scene in which they can select to either proceed to the other asset or exit the game altogether (Figure 12).

Figure 11:
The informative text of the temple Figure 12: Game complete panel Moreover, to make the completion of the game possible, a "ghost" asset is created for each one of the objects (Figure 13). This reveals in transparency the correct placement of each piece, whenever the player presses the right trigger button. The player cannot interact with the ghost asset, its existence is restricted to aiding the player to assemble the objects. Finally, for more realism, the controllers are visualized as red gloves ( Figure 14). When the menu button of the controllers is pressed, a text appears ( Figure 15) that reminds the players which button enables each action that should be taken. This was added because the left and right controllers are used for different actions and players may get confused.

Possible Improvements
After the development of the game was concluded, it was evaluated and tested. Some major issues occurred regarding the functionality of the game and as a result they diminished user experience because they mainly caused fps (frames per second) drops. The issues were detected and located using Unity Profiler ( Figure 16), a tool that provides feedback about the performance of the application in areas such as the CPU, memory, renderer, audio, scripts and garbage collector.

Figure 16: Unity Profiler
In order to resolve these issues, source codecode that has not been compiled yethad to be improved. More specifically, it had to be ensured that the code is not poorly structured, does not make unnecessary calls to other code, is not being called when not necessary or, finally, that it is not too demanding. First of all, code that runs frequently and is found on many assets on the scene, is moved out of nested loops to avoid adding up any inefficiencies. Secondly, it was reconsidered whether parts of the code must run every frame or only when it is needed. In this case, these parts are moved out of the Update function, which is called once every frame, and called upon only when an action triggers them or once every other frame if they cannot be triggered. To set an example, the examination of the completion of the game does not have to be done on every frame, but only when an asset's layer is changed, so there is no reason for this to be done via an update function. Finally, the way the questions appear and disappear changes. At first, each question instantiated every time it had to appear and was destroyed when it was answered correctly or incorrectly. This left garbage in the application which made the frame rate drop, so it was decided to make the canvas that hosts the questions appear and hide from the game view accordingly instead.
The hardware issues are more difficult to fix because the easiest solution is to use a better in terms of performance computer system, but an effort is made to optimize graphics rendering. To begin with, light sources in the scene are reduced and replaced with baked lights. By doing this, Unity calculates all light settings in advance and saves them to the disk as lightning data, so these data are loaded at runtime without having to execute any further calculations. Furthermore, the texture quality of the two assets is reduced from an initial of 4096*4096 pixels down to 512*512 pixels, to make it easier for Unity to load the scene's data. This change does not affect the user's experience, because it is not noticeable in the Virtual Reality environment, even though the distance between the player and the asset is short.

EVALUATION AND CONCLUSION
This research helps to underline the differences between simple 3D computer games and Virtual Reality ones, not only as far as players are concerned, but also for the developers, since they become much more engaged with the application they are creating. Virtual Reality can be exploited in various domains, with Serious Games being the greatest example, changing the way knowledge is obtained. When it comes to cultural heritage, 3D models can enable studying, preserving and reviving it.
It was essential in this application to use the experimental cube, as the 3D models of the assets are very heavy, and it would be more difficult to familiarize with the tools of the game engine and the coding. For the same reason, the surface of the models had to be reduced, otherwise they would not be manageable in the game engine. The game in its final form begins with the selection of one of the two available assetsthe sculpture "Emperor and Barbarian" or the temple of Demeter-, depending on the desired level of difficulty. The next scene provides players with a short text containing information about the selected asset, educating them about its architecture and history. While on gameplay, the players have to break the asset by scattering its pieces in a specific range in the scene, and in order to re-assemble it, they have to answer correctly one "True" or "False" question for each piece they get to move. The game is finished when the fragments are completely reassembled.
The application was tested by a small group of people due to COVID-19 restrictions and evaluated for its usefulness, convenience and ability to create an immersive environment. The users were generally satisfied with the game controls because the actions felt natural even to inexperienced users. The menu that reminded the actions appeared to be a useful but not completely necessary addition to the game, because the actions were not that many and users could easily remember them. Due to the use of teleportation for moving around the scene, players did not experience any dizziness or disrupt during the playtime. Moreover, they could easily grab the pieces and relocate them, with the only exception being some pieces of the temple which were smaller than the others and, since they were placed on the ground, it was more challenging to reach them. The "ghost" was an essential supplement to the game, because without it, putting all the pieces together took much more time, making the game tiring and thus not fulfilling its purpose, whereas the informative texts made the achievement of that purpose easier, as players were able to answer the questions. The biggest disadvantage of the application was that some users found the HMD to be a little heavy and inconvenient for an application that is mainly static. As a whole, the game was considered immersive, interesting and innovative when it comes to learning techniques and users were willing to test similar applications.
Unquestionably, there is room for improvement. First of all, it was noticed that some pieces did not fit together perfectly because of the shape of the collider that enables interactions, since this is a box, whereas the pieces have more complicated shapes. Secondly, it is important to add more questions, to ensure that every piece can be relocated regardless of the mistakes a player may make. Alternatively, questions could be formed as "multiple choice" and not "True" or "False", so as to offer more chances for a correct answer instead of creating a new question for every mistake. Another improvement would be to detect whether the pieces match even outside of the initial position, i.e., if the player comes across two pieces that is obvious they are placed together but are scattered in the scene, they should be able to connect them and then move them as one.
Of course, this research can be extended by adding more 3D models to create more levels of difficulty and also rewarding the players when they complete the puzzle with the 3D model of the asset, for example.
Generally speaking, applications such as the one presented in this work can be utilized as a learning tool in many different domains, from medicine to history and architecture. The only prerequisites are the 3D assets.