INTEGRATION OF BIM AND MOBILE AUGMENTED REALITY IN THE AECO DOMAIN

BIM (Building Information Modeling) is increasingly present in a wide range of applications (architecture, engineering, construction, land use planning, utility management, etc.). BIM allows better management of projects through precise planning, communication and collaboration between several stakeholders as well as facilitating the monitoring of project operations. The emergence of Augmented Reality (AR) technology allows the superposition of (2D, 3D) information directly on the physical world, so generating immersive, interactive and enriching experiences for the user. To take advantages of BIM and AR potential in the interaction and the intuitive management in AECO (Architecture, Engineering, Construction and Operation) projects, we propose a BIM-based AR workflow through an application called "EasyBIM". This latter allows access and interaction with a BIM model through functionalities for measurement, data consultation, collaboration, visualization and integration of information from sensors. The application is developed for mobile platforms (tablet, smartphone), and has as input an IFC file (Industry Foundation Classes). Promising test results show that the developed solution can be easily integrated into a BIM context for several use cases: marketing, collaboration, site monitoring, facility management, etc.


INTRODUCTION
Several new technologies have been proposed to help improving the pre-construction, construction and post-construction (operation) process for building projects. Among these technologies, BIM (Building Information Modeling) has proven high capabilities in supporting various life-cycle activities and assisting managers in decision making (Machado and Vilela, 2020). BIM is widely used in a variety of fields (e.g. architecture, engineering, construction, land use planning, etc.). It allows better scheduling of projects, high level of cooperation between the project partners, optimization of planning's and reduction of cost and time. The BIM process allows improving effectiveness and quick time reaction, thus leading to a comprehensive understanding of the project information during the construction lifecycle.
On the other hand, Augmented Reality (AR) has emerged as an easy-to-learn and easy-to-use method that allows superposition of computer-generated information (2D, 3D or their combination) on real-world objects, which creates an immersive experience with the content being displayed (Kharroubi et al., 2020). This technique minimizes intermediates between the user and the data. While access to BIM models is conditioned by computational power and adequate software, which can be a major limitation for instant use, AR augments accessibility to BIM content and offers an interactive environment with the BIM model. For instance, AR can promote BIM information and improve the efficiency of operations on the job site (Chu et al., 2018). In this context, the use of low-cost mobile devices is very promising to allow more access to AR applications.
Integration between BIM and AR can constitute a reliable tool for coordination and communication in the Architecture, Engineering, Construction and Operation domain (AECO). Furthermore, access to low-cost mobile devices allows increasing the development of Mobile Augmented Reality (MAR) applications and make them available for a large community of users. However, MAR still faces many technical challenges, namely the georeferencing of the BIM model in the field, battery limitations, narrow field of view, low tracking accuracy and uncomfortable device,…etc. (Delgado et al. 2020).
The integration of BIM and AR has recently been a subject of a substantial number of research projects and publications. This research trend tends to develop solutions for improving dynamic access to rich information available in BIM. For instance, Feng and Chen, (2019) proposed a system for assisting construction engineers to perform inspection and progress tracking operations. Garbett et al., (2021) developed a technological solution to reuse the data derived from a BIM to support synchronous collaboration in an AR environment. A systematic review of BIM and AR integration is proposed by Sidani et al., 2021). Their review highlights several limitations in AR implementation, such as connection and localization problems, lack of non-geometric information and other challenges in using AR techniques in the construction site.
Our work tries to show opportunities in adopting mobile AR technologies and identifying gaps within the AECO industry. We propose a workflow for the development of a MAR application for interaction and communication with a BIM model. The solution recreates a reliable BIM experience by giving access to a BIM database that can be utilized in a variety of fields. It serves its purpose of increasing accessibility to BIM, through a user-friendly solution available for mobile devices.
The rest of the paper is organized as follows: Section 2 presents the progress of AR and BIM applications in the AECO domain and offers an analysis of the potentialities of integration of BIM and AR as well as its limitations. Section 3 describes the developed application and gives details about its architecture and functionalities. In section 4, the developed functionalities are detailed and discussed in terms of potential uses. Then, the limitations and the future research perspectives are highlighted. Finally, the paper ends with a conclusion (section 5).

BACKGROUND
In this section, we first introduce the concepts of BIM and AR. Then, we discuss the benefits of integrating both concepts in the AECO domain. Finally, we highlight the limitations and persistent challenges of such integration.

Overview of BIM and AR
The concept behind BIM has existed for a long time since the mid-1970s (Eastman and Others, 1974). The term BM (Building Model) was used for the first time in a mid-1980s paper (Ruffle, 1986). But the BIM concept, as used today, first appeared in 1992 (van Nederveen and Tolman, 1992). Today, BIM has garnered prominence throughout the AECO industry and has found a wide range of applications (Moreira et al., 2021). BIM is generally defined as the process of generating and involving a digital representation of a building or construction and its characteristics (Noghabaei et al., 2020). Through BIM, the project's stakeholders can work collaboratively and simultaneously, so minimizing the risk of incompatibilities.
BIM has driven a significant change in the AECO domain.
Particularly, BIM provides predictable information regarding quantity, cost, schedules and materials (Chen et al., 2013). BIM serves as a central data repository that can store information about a facility for managing the lifecycle of a construction project from the design to the maintenance phase. Indeed, BIM facilitates information flow since participants have a clear understanding of their role and responsibility in a project which allows aligning expectations and increasing transparency.
Beyond the production of 3D models, BIM can be used for different functions such as improving communication, enhancing decision making (by simulating different construction scenarios) and ensuring a good visualisation of a project (Noghabaei et al., 2020). However, the latter aspect of visualisation still presents one of the inherent shortcomings of BIM (i.e., lack of interactive visualisation and immersion using a keyboard and mouse in a 2D screen). To tackle this inherent difficulty of in-situ BIM visualisation and make advance in AECO (such as real-time on-site communication), researchers have proposed the use of AR which involves the enhancement of the real world by overlaying virtual objects via a computer. Indeed, BIM allows early access to information, this fact together with the ubiquity of mobile technology like AR, shortens the gap between information availability and response times.
AR is a technology that allows overlapping information and computer-generated graphics to real-world images or models. AR makes it possible to combine a real environment with computer-generated information, developing a 3D space in which generated computational elements are superimposed on the user's real field of vision (Wang et al., 2013a). Two different types of AR are considered: 'Video See-through' and 'Optical See-Through'. The first one uses monitor-based devices (e.g. mobile phone, tablet), while the second uses headmounted devices (e.g Microsoft HoloLens). Besides, AR applications can be classified into two categories: 'markerbased AR' and 'markerless AR'. The first one has to use a marker as a trigger, while the second normally uses localization methods to link the virtual world with the real world (Kharroubi et al., 2020). AR is beneficial in several domains. Besides applications in architecture, engineering and construction (Behzadan and Kamat, 2011), there have been several AR applications focusing on Facility Maintenance Management (FMM). For example, Irizarry et al. (2013) proposed an AR system for facility managers to access maintenance information, which has proved high capabilities in improving efficiency during FMM. Furthermore, Alam et al. ( 2017) integrated AR and VR (Virtual Reality) with the Internet of Things (IoT) to improve safety and reduce errors for FMM activities. Similar applications have also been developed to apply AR to FMM Liu and Seipel, 2018).

Integration of BIM and AR in AECO
In the last decade, AR has received considerable interest from the AECO community. Indeed, AR could represent the site extension of the BIM concept and approach, and so maximizes the potentials of BIM in the construction site. AR is deemed to be a key enabler to address the current shortcomings of BIM onsite use in construction (Raimbaud et al., 2019).
The integration of BIM and MAR in-situ presents a plethora of improvement in visualisation (Yuchen, 2017), interaction and communication among stakeholders (Chalhoub and Ayer, 2019). The in-situ option offers the possibility information about 'as-built and as-planned progress' as well as 'current and future progress' can be obtained and presented visually. It has been revealed that the use of BIM and AR can improve building management and construction procedures. For instance, visualisation of the construction associated with the planned model can improve the identification, processing and communication of progress discrepancies in the construction site. AR can promote BIM information and improve its efficiency at several levels. It ameliorates the efficiency of operations on the job site (Chu et al., 2018), enhances BIM at the visualisation level of the building site (Machado and Vilela, 2020) and provides relevant information for maintenance works . Furthermore, AR can be implemented at the design phase to navigate through the design options during collaborative meetings (Ammari and Hammad, 2014). During the construction stage, the usefulness of the BIM of the project can be increased through AR application. The purpose is to reduce time in the schedule, minimize costs and ensure the quality of the product through an improvement in the constructive process. It is also possible to associate a temporal dimension to the BIM (BIM-4D) to make a comparison between what is being executed and what was programmed in real-time (Han and Golparvar-fard, 2014). Besides, the automation of progress monitoring is important since early detection of fallout in schedule represents an opportunity to decrease the impacts on time and cost.
Three mental aspects need to be addressed when assessing the feasibility of using AR for construction-related work processes (Dunston and Wang, 2011): 1) Information searching and accessing which relates to how information is obtained; 2) Attention allocation which relates to the distraction from other tasks and 3) Memory which relates to sensory, short-term and long-term memory function. Based on the aforementioned aspects, Wang et al. (2013b) proposed a conceptual framework for integrating BIM with AR which comprises three layers: 1) BIM, 2)AR tracking/sensing for context-aware, and 3)AR visualisation/ interaction. The authors presented a series of examples to describe how AR can be used for reasoning in the interdependency, spatial site layout collision analysis and management, link digital to physical, project control and visualisation of design during production (Wang et al., 2013b).

BIM-AR limitations in AECO
AR technology has been proposed as a way to extract information from BIM models to improve the efficiency and effectiveness of tasks performed by the project partners. More and more low-cost mobile combined AR and BIM tools are developed to access information facilities (Irizarry et al. 2013). However, the level of AR adoption is still low. This is due to some limitations (Delgado et al. Construction projects are delivered by a large and varied group of small companies and the supply chain is highly fragmented.
To participate in addressing the aforementioned limitations, we present in this paper a framework that integrates BIM and MAR within a low-cost application based on mobile devices to enable the physical context of each construction activity or task to be visualized in-situ in real-time and to allow interaction with the associated BIM model. Our contribution provides also a proof concept to respond to the high-level requirements of the professional sector. We identify, also, which device capabilities need to be improved for "the engineering-level" in-situ use.

METHOD
In this paper, a MAR application named "EasyBIM" is developed for interaction with BIM models. The workflow is composed of three main steps. First, the application architecture is defined. Then, a configuration of the AR environment is performed and finally, functionalities are developed. The consecutive steps are discussed in detail in the following paragraphs.
The BIM model (Figure 1) used for this study is downloaded from (https://www.coursera.org/learn/bim-fundamentals). BIM information for each "BIMObject" is available as a C# dictionary. After synchronization, the BIM model is available in Unity 3D environment and is structured according to IFC (Industry Foundation Classes) definitions which are represented by 3D GameObjects. The application development is released by the use of AR Foundation Augmented Reality API (Application Programing Interface).
• Scene1: includes some functions for the interaction with the BIM model such as plane projection, rotation, scaling, layering as well as displaying BIM information related to selected objects. • Scene2: offers a notification system that centralizes all information relative to the project, a planning visualization system associated with the model and a 3D measurement tool.
• Scene3: allows collecting real time information from sensors linked to the model and then visualizing the information in an AR environment in the form of 3D graphics.
The development of the application requires downloading the following extensions: SDK (Software Development Kit), NDK (Native Development Kit), JDK (Java Development Kit) and the "Android Build Support". For each scene, three sessions are fundamental: • AR Session: this session coordinates the processes of "AR Foundation" to create an AR experience. It establishes a correspondence between the real objects and the virtual environment for the environment tracking; • AR Session Origin: this session allows transforming the detected objects between the local system of the camera and the coordinate system of Unity; • AR Camera: It allows real object tracking and the generation of rendering to be superposed to reality. The "Tracked Pose Driver" permits tracking the position and the rotation of the camera to be integrated with the «AR Session Origin ».

RESULTS AND DISCUSSION
In this section, we expose the developed functionalities for each scene and discuss its benefits in the interaction with the BIM model.

Scenes functionalities
Scene1 (Visualization): The application uses plane detection (horizontal and vertical) through the "AR Plane Manager" script to track available planes for the user to display the model. The "Raycast" function is utilized to place the model conveniently.
Scaling and rotating the model is also possible through dedicated sliders, making it a tabletop application and also a tool for virtual visits. AR "Raycast" triggers information contained in each 3D geometry and displays it through a table on the screen. The model is also structured into customizable layers (walls, structures …). This is possible through a loop looking for objects with similar names from the 3D Objects database. For each selected object, related information can be visualized (Figure 3).
Scene2 (Measures and collaboration): Taking measurements on detected planes is a useful tool. This functionality places tiny spheres on user screen touch and saves their 3D coordinates, the line renderer is then utilized to draw a line between the two entities, which gives a realistic measurement sensation. A 3D "TextMeshPro" is used to display dynamically the distance on the 3D line (Figure 4).

Figure 3. Visualization of BIM information related to each "BIM Object"
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-4/W3-2021 Joint International Conference Geospatial Asia-Europe 2021 and GeoAdvances 2021, 5-6 October 2021, online Furthermore, the application allows a collaboration functionality to permit the user notifying information relative to the BIM model (updates or eventual detected errors) to the project team or the BIM manager. These notifications can then be accessed through a notification dashboard displaying all the notifications. Thanks to a connection between Unity and Google Forms, Google Datasheet. Admins can directly access the Google Sheet Database. Meanwhile, users can access filtered information directly through the application's user interface. The notification process is illustrated in Figure 5.
Scene 2 integrates also schedule management. A schedule defined as a ".csv" file according to BIM planning standards (i.e Navisworks Timeliner) is displayed as a table directly on the BIM application as Unity 2D tables. We propose a special button dedicated to looking for updates saved in the Google Drive database. Apps are then named by date to keep track of the latest version and can be directly downloaded through the application. However, real-time updates of the BIM model are still a challenge. This scene allows visualization of information from sensors. Sensor data are stored in a datasheet database and then integrated into the AR application. We can access information about temperature, humidity and a calculated risk score from the two first information. After plane detection, cubes are placed in the AR environment. Through a 3D scaling according to the value of each parameter, the cubes help visualizing data contained in the database by adding 3D text on top of the scaled cubes (Figure 6). The information is real-time updated when new information is available.
Notification Table   Notification Google sheets Data Input Visualization Figure 5. The notification workflow The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-4/W3-2021 Joint International Conference Geospatial Asia-Europe 2021 and GeoAdvances 2021, 5-6 October 2021, online Figure 6. Visualization of real-time information from sensors.

Discussion
The application developed in this paper addresses the technical challenge about the integration of BIM and AR in the context of AECO. It's based on developing some useful functionalities for the interaction and the dynamic flow of information between the user and the BIM manager within an AR environment.
Thanks to this application, efficiency of operations on the job site is enhanced (by visualizing and updating information about BIM objects), information about the building performance is accessible (through real time data from smart sensors), and collaboration and schedule management for a project is facilitated. Furthermore, the application is low cost and easy to use and can so be largely adopted within the AECO community.
Compared to other existing applications, EasyBIM is open access (the code is available on demand) and modular. It can so be enriched with other functionalities to fit to specific needs related to its use in several contexts of AECO projects. Furthermore, many applications on the market are «BIM Viewers » where AR is mainly used to enhance the user experience. In the case of EasyBIM, apart from visualization, the developed scenes allow collaboration and updates of information related to BIM objects. The notification system allows the BIM manager to be informed at time of any updates of the 3D mock up. Besides, the schedule management of a project makes a significant difference with most BIM AR tools in the market.
EasyBIM addresses also the operation phase of a building by integrating access to real time data from smart sensors. This added function makes the application multi-use and extends its adoption to Facility Management by adding other functions for managing buildings installations and equipements. This widens the scope of use of the application at the different stages of the building from construction to operation while existing developments in the field mainly address a specific use case related to a certain stage of a building project.
"EasyBIM" application brings together a wide range of functionalities that can coexist and complement each other around the same project. It's now generic and can be later adapted to add other functionalities dedicated to some specific contexts. We present below some perspectives of uses in real estate market, site work progress and Facility Management: Real estate market: In this context, the application can give a deliverable widely used in the case of VEFA (Sale in the Future State of Completion), thus replacing the traditional 2D plans. Easy to use and more intuitive, this application combines the architectural aspect as well as a panoply of information related to the various objects which may interest customers.
Site work progress: The first scene gives access to BIM data that can be used in-situ to facilitate communication around a project. The ability to view the model at scale (1:1) is a major advantage, although georeferencing has not yet been implemented (discussed in the next section), rough calibration is still possible. This application will have its interest in the facilitation of work in general. The model coupled with the notification system allows easy sharing of data between all partners, the entry of comments in the field and direct comparison with the model allows this process to be optimized.

Facility Management: EasyBIM offers a connection with an
IoT system allowing real-time visualization of data from sensors. The use of AR makes it possible to overcome any limitation relating to the screen in the case of several sensors, for example. Although our approach assumes that data integration is already done at the level of the Google Drive database, there are other methods to integrate IoT data into BIM through open standards (Moreira et al., 2021).
Several important issues should be also pointed out regarding the BIM development. The georeferencing of the BIM model remains however problematic, which is due to the imprecision of the smartphone sensors.
As future work, we plan to test potential solutions for addressing BIM georeferencing issue on site such as MapBox, SiteVision of Trimble, etc. (Mustaqim et al., 2019). We will evaluate the quality of this georeferencing by proposing adapted metrics that take into account local and global georeferencing. Furthermore, to be able to use the model together with the reality on the ground, the occlusion factor must be taken into account. One solution is to vary the opacity to make the model and the underlying object appear at the same time. Choosing the correct transparency is crucial to highlight the desired objects while omitting others. The second solution for the occlusion can be tested by using the Lidar (Light Detection And Ranging) system integrated in recent mobile devices (Gugliermo, 2019).

CONCLUSION
Our work brings a solution based on an integrated BIM-AR framework through the development of a MAR application for managing information of BIM models. The developed functionalities allow distance calculations, manipulation of the model, visualisation of information linked to each BIM object, scaling /rotating of the model and real time visualisation of information from sensors. The application brings a proof of concept to respond to the requirements of the construction sector which is high demanding in such functionalities. Furthermore, it can be valuable to integrate more functions to deal with some specific use cases in the AECO domain.
We argue that two main technical problems are to be addressed as a future work: 1) the registration problem, which relates to the spatial alignment of real world and AR environment and 2) The occlusion problem, which relates to the required visual illusions so that the virtual and real-world coexist in a credible manner. Furthermore, developing functionalities for updating the BIM model within the AR environment is still a challenging issue.