Possibilities of spatial correlation of 3D models in an archaeological augmented reality application

The use of Augmented Reality (AR) technology is widespread in countless archaeological sites and a variety of applications. Archaeological excavations lead to archaeological finds, some of which are transported for preservation and then for exhibition in museums (jewelry, vases, etc.), while another part of them is documented in detail and remains in situ (eg building walls), roads, grave covers, etc.). However, after the registration of the archaeological finds, it is impossible to observe them. As part of our research project, we will develop for the first time AR methodology and procedures for the observation of covered archaeological finds on mobile devices (smart phones, tablets), which were registered after their documentation. AR technology in recent years has seen great growth in terms of implementation platforms and available software, as well as the tools developed to support it. These tools either make their appearance in the form of frameworks, extending the capabilities of an existing engine, or function as independent services. At the same time, progress has been made in the field of sensors of mobile devices, which makes the compatibility of hardware and software another issue to be researched. As part of the development of the above application for mobile devices, an evaluation is made of the most widespread AR Frameworks that support the Unity3d Game Engine and the compatibility / interoperability with the sensors of different categories of mobile devices. The frameworks were checked and evaluated for placement and tracking of the positions of the 3D covered objects. In this paper also, methodologies and techniques used in space detection and tracking are presented and evaluated.


INTRODUCTION
Many countries host in their underground a plethora of ancient remains, many of which are known to us through excavations carried out at the respective sites. These remains are of great archaeological interest, as they contribute to the world heritage and become an object of great interest. In this context, a large number of visitors come annually to the places where archaeological excavations have taken place, thus demonstrating the special influence of these sites in the tourism industry. However, many of these remains after their registration are reburied in order to avoid exposure to the external environment and as a result, they are no longer visible, which can degrade the tourist tour experience. The problem of invisibility of ancient remains seems to create a fertile ground for the use of Augmented Reality technology (Figure 1), as it promises realistic and oriented three-dimensional representations of virtual objects in the real surrounding archaeological site. In recent years there has been a rapid development of Augmented Reality technology both in terms of its technical progress and its dissemination. The term Augmented Reality refers to the technology which, through the appropriate hardware and software, succeeds in enriching the surrounding space, the image of which is taken by a camera and projected on a screen, with virtual digital objects. These objects are influenced by the actual environment in terms of orientation, size, distance from the user and in many cases by lighting. Augmented reality is a promising technology and has been researched for more than 50 years. However, like most emerging technologies, it was not always growing at the same pace. In the earlier years of Augmented Reality technology, the equipment required to develop and market an Augmented Reality application made cost, time, and lack of device flexibility impediments to its development and availability.
Specifically, the literature reveals that the first Augmented Reality device was built in 1968 by Ivan Sutherland under the name "The Sword of Damocles", which consisted of a portable screen, a bulky arm and the main body of the device. Now, the conditions in terms of hardware and software are more favorable than ever. Every year more and more powerful computing devices are available on the market, which are equipped with a set of sensors, such as gyroscopes, accelerometers and TOF lenses. At the same time, due to the more powerful processors that these devices have, it is possible to run more demanding algorithms and process information from more subsystems of the device, resulting in better performance and quality in terms of Augmented Reality. It is also worth noting that while the quality and computing power of the material is constantly increasing, its availability and price are relatively stable, showing some fluctuations in different periods. All of the above conditions, as far as the hardware part is concerned, obviously affect the software part to support and develop the Augmented Reality applications. In this context, major gaming machines such as Unity3D and UnrealEngine have chosen to support Augmented Reality application development platforms, making this technology more popular and accessible. In the framework of this research project, the aim is to explore the available tools for the development of Augmented Reality applications, to evaluate them and finally to implement an application for mobile devices such as mobile phones or tablets, which is suitable for use outdoors in cultural heritage. The bibliography identifies research efforts related to Augmented Reality technology, aiming at the digital representation of various outdoors cultural heritage elements ( Figure 2). These efforts focus either on evaluating the available Augmented Reality technologies, or on implementing an application that utilizes them. In the context of our present work, more emphasis was placed on research attempts which refers mainly to applications of Augmented Reality that were implemented in order to be used outdoors. In those efforts, interesting approaches are presented both at the level of implementation and at the level of application. In addition, it is important to consider the problems and the ways in which they are dealt with in each case. In the tests performed in the present work, it was realized that all Augmented Reality frameworks operate with an inherent weakness in the lack of visibility, as most space tracking methods rely to some extent on camera visibility. This results in applications that are implemented and supported by these frameworks having to deal with this vulnerability, as they cannot work without the necessary visibility. This issue is crucial for the performance of an Augmented Reality application, especially when its user space is outdoors. In tests performed, it was observed that outdoor lighting conditions are often unpredictable as factors such as cloudiness, shading and glossy surfaces can significantly affect the performance of the application. However, similar weaknesses occur in other environments, such as in areas that are not adequately lit or below the water surface. An interesting approach to this issue is to track the user through an underwater audio system [Fabio Bruno et al., 2019], which has the ability to record the user's path in 3D. Alternatively, both marker-based and markerless methods have been proposed. Marker-based methods [Cejka et al., 2018] are often presented in conjunction with input data correction algorithms [Sola 2017, Neunert et al., 2015 due to partial or total lack of visibility or potential errors that may occur in inertial system of the device. For markerless methods, solutions have been proposed that utilize either the device's inertial system or, in more specific cases, an acoustic positioning system. In addition to the technical solutions that are proposed, it is important to consider the solutions that are related to the design, use and application of Augmented Reality technology in cultural heritage sites. In particular, solutions have been proposed [Damianos Gavalas et al., 2016] which include the display of points of interest through the camera of the device using two-dimensional oriented virtual elements which behave according to the distance of the respective point of interest, which they represent from user. Through this approach, the user is guided in the space and informed about the surrounding points of interest, while maintaining a good enough level of interaction to arouse the user's interest. In addition, it is important for the user through the application to have at their disposal a floor plan or a map of the space he is navigating [Rozhen Kamal Mohammed-Amin et al., 2012], as this makes it easier to choose a path for his navigation and shorter the time it takes to orient. In addition, mobile devices commonly used in Augmented Reality applications are one of the ideal platforms for presenting multimedia material such as image, text, audio and video. This feature of mobile devices can be used in the development of an application to condense information, to enrich the application aesthetically and to maintain the interest of the user.

PRESENTATION OF AR FRAMEWORKS AND HANDHELD DEVICES TYPES.
In an effort to evaluate existing Augmented Reality tools supported by the Unity3D gaming machine, the most popular and widely available frameworks were collected and tested. The evaluation process involved the creation of small projects (at least one for each) which utilized at a basic level the capabilities of each framework. The above software was tested in the version of Unity3D 19.2.17f1 and evaluated in the methods marker-based AR tracking and markerless AR tracking and as criteria were set the stability of the tracking, the immediacy of the response, the price of the framework, the ease of application development with the corresponding software and the restrictions that each product introduces. The evaluation of the frameworks was done after the implementation of small Augmented Reality applications and their testing with different devices. Specifically, in each of the examined Augmented Reality frameworks 1 or 2 small applications were designed and implemented to test their capabilities in practice. It is important to mention that the frameworks that were examined were not exhausted in terms of their overall functionality, but in terms of the 2 basic methods of tracking. The criteria used to evaluate the frameworks were the quality of the tracking, i.e. how steadily and without errors the reference points are tracked, the immediacy of the camera response and the application as a whole when using a framework and finally the overall quality of the user experience from the point of view of application development, for example how easy it is to debug in the respective framework. The reason why these criteria were used is to identify the strengths and weaknesses of each Augmented Reality framework, so that on the one hand the most appropriate framework is selected for the implementation of an Augmented Reality application, which can be used in outdoors cultural heritage sites and on the other hand. to highlight the issues that need to be addressed when developing such an application. At this point, it is important to clarify that the following Augmented Reality frameworks have been evaluated for their relevance to this project and that the following evaluation meets only these needs and therefore the ArFramework A-Z designations were used instead of their real names. In addition, in terms of presenting the results of the evaluation, it was considered important to refer to the price range corresponding to each framework. The evaluation values range from 1/5 (unsatisfactory) to 5/5 (very satisfactory). Below there will be an extensive presentation of ArFrameworks (Table 1), as well as an analysis of the features they offer and the limitations they display.  In addition, it is obvious that the performance of frameworks depends to a large extent on the devices on which they are used. For this reason, the devices used for the tests are listed below ( Table 2). The aim of the selection of these devices was to cover the widest possible range of prices and market possibilities.

AUGMENTED REALITY TRACKING METHODS
From the first stages of the research program, the need arose to investigate the methods by which the tracking of the position of the device in relation to the surrounding area is achieved, in order to apply the technology of Augmented Reality. The areas of investigation of tracking methods include the description of each method, the recording of the advantages, disadvantages and limitations introduced during the development and use of an Augmented Reality application.

Marker-based tracking methods.
Marker based tracking, is the most common method used in Augmented Reality applications (Figure 3). In addition, it is a method that is mainly based on image recognition technology. This leads to the need to create a dataset of metadata, which arises from the processing of the original image formats and are intended to be used as Augmented Reality markers. In most Augmented Reality frameworks, in order for an image to be recognized as a marker by a device's camera, a preprocess of it is required, in order to be built in the marker dataset of the application. The marker dataset in most frameworks is created by some external software and then copied into the application files. That dataset is then used during the execution of the application. Specifically, as long as the camera is utilized by the application, the dataset data is constantly compared with the corresponding data received by the camera. During the execution of the application, when a large percentage of similarity is detected between some data of the dataset and the data entering from the camera, then the process of placing the virtual object in space is initialized. From this point until the marker is not within the scope of the camera, the Augmented Reality framework constantly adjusts the position of the object in space in order to give the feeling of the mixing between virtual reality and reality. The main advantage of this method is the stability and quality of the tracking it offers, in relation to the restrictions it introduces. Specifically, as long as the camera of the device has the required visibility and after the appropriate dataset with the markers has been created, this method is able to offer a decent Augmented Reality experience even with older or low-cost devices. This method also does not depend on the sensors of the device since the only input device it uses is the camera. It is worth mentioning, however, that some frameworks such as ArFramework Z have the option to use the available sensors of the device supportively in order to improve the final result, although it is an exception. However, it is important to mention the disadvantages of the marker-based tracking method. This tracking method is based entirely on the camera of the device. This leads to instability or inability to detect the markers in case for some reason the visibility of the camera is altered or reduced. In practice, the factors that may lead to such an effect may be the partial or total lack of illumination, excessive illumination or illumination on a smooth surface at an angle such as to create intense shine on the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China surface of the marker. In addition, if the marker is not within the camera's field of vision, Augmented Reality is impossible to be achieved. This reduces the feeling that the virtual object is actually in three-dimensional space, especially at the limits of the camera's field of vision. The drawbacks of this method are not limited to issues that arise during execution. In particular, the mandatory use of external software to create the dataset with markers, is a limitation that does not exist in other tracking methods.

Markerless tracking methods.
The markerless tracking is also a widespread and promising method, which is constantly gaining ground. This method is more environmentally independent than the marker-based method, as it does not need a specific location reference point like a marker. In most Augmented Reality frameworks, the method is applied by planar detection (Figure 4), which is used as a reference level in space. Planar detection methods may differ between Augmented Reality frameworks. Some frameworks such as ArFramework C have the ability to use more than one planar detection method depending on the device running the application. The well-known planar detection methods in their simplest form, involve the use of device camera, gyroscope and accelerometer. In this case, the camera is used to detect the level, while the gyroscope and the accelerometer are responsible for constantly sensing the rotation and displacement of the device and transporting it to the virtual environment. It is worth mentioning that in case the device supports ArCore or ArKit technology the result is significantly improved. More complex level detection methods are those that, in addition to the camera of the device, utilize Lidar technology and ToF lens technology, which, however, are not investigated in the present work. A key advantage of this tracking method is its independence from environmental factors such as shading and polishing. During development and execution. In particular, unlike other methods, no pre-processing is required as the algorithms used operate exclusively in real time and use the data received from input devices such as the camera and the various sensors of the device. This independence results in a more consistent version of Augmented Reality. Specifically, with this method in cases where the camera instantaneously does not have good visibility, it is observed that the Augmented Reality performance is not affected as it does not depend solely on the visibility of the camera.
In contrast to its advantages, this method introduces some limitations. In most of the frameworks investigated in this work, it was realized that level detection can only be done by specific devices. ArCore or ArKit support, in particular, is a common requirement for a device in order to be used to run a markeless Augmented Reality application. As a result, these methods are less accessible to a large number of users and developers. In addition, such a method, despite the consistency, the independence and the flexibility it offers in terms of the environment, the visibility of the camera and the development of an application that uses it, shows reduced stability when running an application. Lack of stability is often manifested by a momentary misplacement or abrupt displacement of virtual objects in space and drop in the framerate of the application. At the same time, the lack of stability is perceived in cases where a sensor has a low sample rate or it gives momentarily wrong values.
As part of the evaluation of the Augmented Reality frameworks, some additional limitations were introduced, which are independent of the tracking method used. In particular, instability of the projected material was observed in the abrupt movements of the device. The inability of the device to follow abrupt changes in its position is also found in the aforementioned methods, as the factors it causes are related to the hardware of the device or parts of it, such as the sensors or the camera. In addition, another issue to be explored is the application of occlusion, the absence of which is evident in Augmented Reality applications ( Figure 5). The term occlusion refers to the hiding of distant objects in space from the nearest objects, which does not occur between virtual and real objects. This feature is a key issue for Augmented Reality, as tests have shown that a lack of sense of depth when viewing virtual objects reduces the quality and realism of the result.

CONCLUSION
In this paper, examples of the application of Augmented Reality technology in cultural heritage sites were presented. The aim of these applications, is to virtually revive the cultural heritage sites and to provide assistance to the user in orienting him to places where there is a partial or total lack of visibility of points of interest, through the technology of Augmented Reality. These applications are installed on mobile devices, such as smartphones or tablets and are used mainly by tourists, in order to enrich their browsing. The virtual objects that are placed in space through the technology of Augmented Reality, in most cases are two-dimensional graphic elements or threedimensional models that function, either as points of interest, to orient the user more easily, or as additional objects in space, which enrich the user experience, thus reviving the surrounding space.
In this paper is also covered extensively the most popular Augmented Reality frameworks supported by the Unity3D game engine. This report contains data collected experimentally from a series of small projects tested on devices, which belonged to a wide range of prices. In addition, a rating of each framework is provided in terms of site tracking quality, response, user experience and price. The evaluation of Augmented Reality frameworks was mainly based on their overall performance in the 2 most common methods of tracking, which are applied in most cases, thus creating the need to make an extensive reference to these methods and to evaluate the advantages and disadvantages of each of them, are noticeable by end-user. Spatial tracking methods can be distinguished into marker-based methods, which are both the reference point in space, and markerless methods, which usually use planar detection algorithms, in order to have a reference point in space. Although Augmented Reality is a promising technology, examining the frameworks and methods available to the public has revealed some serious weaknesses in this technology, which either introduce some limitations or degrade the end user experience. The main weaknesses identified are the lack of occlusion, which reduces the quality of the visual effect by not giving the right sense of depth, the lack of stability in tracking, which often becomes apparent when using an Augmented Reality application, especially on devices which belong to a lower price range. A significant limitation also is that lower-end devices are, in most cases, not fully supported by Augmented Reality frameworks, as a portable device that does not support ArKit or ArCore cannot use the available tracking methods. not based on markers. This leads to a reduction in the range of devices that can support the full functionality of an Augmented Reality framework, thus increasing the cost of both the equipment and the development of an Augmented Reality application.

ACKNOWLEDGEMENT
This paper has been accomplished on behalf of the project SMART EYE: Development of a prototype for smart visualization of invisible antiquities. The project is running under the framework of the Operational Program Competitiveness, Entrepreneurship and Innovation 2014-2020 (EPAnEK), Special Actions "Aquaculture" -"Industrial Materials" -"Open Innovation in Culture", Τ6ΥΒΠ-00023.