DATA FUSION FOR CONSTRUCTION MONITORING: HOW LEICA INFINITY MANAGES IMAGES, GNSS DATA, TERRESTRIAL SCANS AND BIM DATA TO EFFICIENTLY TRACK COMPLEX CONSTRUCTION SITES

This work explores the importance of properly handling the big variety and amount of measurements data that nowadays can be more and more easily acquired. In fact, in the last decades a new challenge took hold: the academic and industrial focus, in the most different fields, extended from developing new technologies to acquire data and more accurate ways to process it, to also developing software environments to properly handle and process the big amount and large variety of available data. In surveying, when the size and the complexity of the site requires the acquisition of diverse data and the usage of different kind of devices, working in different coordinate systems and operated by multiple people, data process and interpretation is much more error prone. When surveying a complex construction site, early identification of deviations from the design during the construction phase is crucial and has as well a big impact on the project's economic budget. Building Information Modelling (BIM) is the accepted standard for managing information for an asset, and for the construction phase it provides working from a common visualisation of the as-planned model in a 3D environment. The as-built model is generated from a large variety of acquired data, especially when site complexity requires a high level of data integration. Data are usually acquired using GNSS receivers, Terrestrial Laser Scanners (TLS), Total Stations (TS), Digital Levels, and Aerial and Terrestrial photogrammetric devices. Specifically, over the last decade Unmanned Aerial Vehicles (UAV) have been proving their importance also in construction progress and quality monitoring, thanks to their relative easiness of use and the overview they can provide over the construction site. In this work, the value of merging and processing in a single software environment different data sources to reduce errors and get geodetically relevant data is analysed. Specifically, Leica Infinity, a specialised software able to bring various data sources together for working in a common reference frame, is investigated and its effectiveness and efficiency proved for monitoring a complex construction site. The advantages of an integrated data acquisition and process, and in particular the contribution of UAV data in construction site monitoring, both from an accuracy and completeness standpoint, are also assessed. * Corresponding author


INTRODUCTION
In the last decades, a new challenge took hold: the academic and industrial focus, in the most different fields, extended from developing new technologies to acquire and create data and implementing more accurate ways to process it, to also developing software environments to properly handle and process the big amount and large variety of available data (Staegemann et al., 2021). The interpretation of the captured data, and the correct understanding of the context and its connection with other data, is and has always been a crucial point for the most diverse fields in the scientific research (Martin, 2021). This is true also in the construction and surveying fields. Each step of the various construction phases requires a careful and well executed data acquisition campaign to maintain an accurate monitoring of the site progress. This gets even more important for complex construction sites, where the size of the area or the complexity of the structure add difficulties to the construction's activity, requiring simultaneous acquisition of heterogeneous data, usually in different reference frames, to get a reliable overview of the works' progress. In such cases, early identification of deviations from the design during the construction phase is crucial and has a big impact on the project's economic budget. Building Information Modelling (BIM) is the accepted process for managing information for an asset and involves sharing and visualising the as-planned model in a 3D environment, as well the various measurement validation tasks, and it's widely used as reference to effectively monitor the site over the many phases of construction (Nawari, 2012). The as-built model is generated from a large variety of acquired data, especially when site complexity requires a high level of data integration. Data are usually acquired using GNSS receivers, Terrestrial Laser Scanners (TLS), Total Stations (TS), Digital Levels, and Aerial and Terrestrial photogrammetric devices. Specifically, over the last decade Unmanned Aerial Vehicles (UAV) have been proving their importance also in construction progress and quality monitoring (Kielhauser et al., 2020). As acquiring data becomes easier and more efficient, and data acquisition automation is also finding its way in construction processes (Alizadehsalehi et al., 2018), the amount of available data is therefore growing over time, and the biggest challenge is properly handling the acquired data, fusing different data sources together, processing them, and getting valuable and accurate information to be used in the construction monitoring. Using a software environment able to combine effectively the data sources to work in a common reference frame is therefore crucial.

METHODOLOGY
The goal of this work is analysing the advantages of using an integrated and geodetically relevant software environment to monitor complex construction sites which require a high level of data integration. The reasons of using different data sources to efficiently and fully monitor a complex construction site, where data captured from different points of view and giving different levels of details contribute to the needed overview of the works' progress, are also shown. An important aspect towards accurate results is having the ability to access and combine each of the data sources supporting processing steps. In the following it is also presented how Leica Infinity software can support the monitoring of a construction site, effectively and smoothly managing various data sources acquired in different reference frames.

The value of fusing multiple data sources
The measure tasks require the ability to use the appropriate measure tool. The various measurements from different sensors only have a significance when they relate to a commonly known data value. Working on a construction site requires detailed coordination with various aspects of time and location for the tasks where measurements are needed, for instance marking a location, confirming an installation or validating the results meet the specifications. There are also physical constraints, such as measuring interior and exterior aspects, which require different devices and enable the use of distinct reference frames. These complementary sets of data must relate each other, and it's therefore needed the ability to combine and utilize diverse data sources for accomplishing the objectives more effectively. For a construction work, various sensors are utilized and contribute to the different phases form start to end, from establishing the control network of the project site to creating the as-built model and validation of structures.
A network of post processed GNSS observations observed using data of the local GNSS reference network can establish the site control. Typically, when this is completed, the other phases of construction are relying on it or needing to densify the network within the structures. Not part of this paper but included for the importance of a common control network, would be utilizing the model of the ground and the utilities' infrastructure with the dirt moving machines and a software positioning system for guiding the jobsite for construction foundation work. The same GNSS reference network for positioning of machines can be used. Points relevant for the construction can be staked out using GNSS RTK devices. GNSS RTK data can be also used to acquire control points to be used as survey points and in the image processing as Ground Control Point (GCP).
UAV drones play a valuable role in site monitoring during all phases of a construction project. Establishing the dirt work and foundation phase, imagery is an efficient tool for bringing visual progress overview. It also plays a role for providing a record of the site progress and dirt movement, including deriving a Digital Surface Model (DSM) enabling volume calculations. Using the drone within the GNSS reference network and in the same coordinate reference frame, enables easy data validation and combining of data that can be shared to the machines if required or part of general documentation. In the case when GNSS data of the UAV track is also available, this can be postprocessed, and images' positions updated to get the most accurate possible solution in terms of positions of the images and therefore of reconstructed 3D model.
During the foundation or establishing of pylon structures, in many cases a precise digital level is the tool to provide height readings that would be used for the basis of all structures of the site. Here the different data sources are not dependant to time but have value for working in a project providing data transparency.
Terrestrial laser scanners provide a fast method for capturing a scene that will be used in the office software. For a construction project environment, the scan data is used for validation of interior structure assemblies for clashes or even for flatness checks.
Total station is used to acquire high accuracy control points that can be used both as GCPs for UAV image processing and as target points for the scan registration process. They are also used as a primary instrument to stake out points where accuracy is critical.
All the mentioned devices complement each other towards the project's objectives, having different purposes and working accuracy ranges, as shown in Figure 1. In Section 2.2 it is explained how Leica Infinity office software provides the tools to manage the combined data both from design information and measure information on the common reference frame, or the project control network. Depending on the project site, this will require being aware for and considering how the different measurement systems arrive at the same coordinate system with minimal distortions. In a projected coordinate system, the effect of scale can be centimetres across a job site compared to plane coordinate measurements on the ground. Being able to combine measurement sensors including total station measurements with GNSS data means the projection scale factor is properly considered. This is as well important to fit with the BIM model.

The value of a single processing software environment
In construction projects where multiple sensors are used for measuring tasks, an important aspect to work efficiently is having the ability to access and combine each of the data sources supporting processing steps. It has been already mentioned that measurements are only useful when related to a commonly known point. How each measurement is referenced to the known depends on its technology. GNSS sensors work from a world coordinate system WGS84, and they are not practical for construction work without projecting to a common reference system. Total stations require a known position and orientation in the common reference system to relate the acquired observations to it. How the measurement will be utilized is then defining the workflows both in field acquisition and data processing in the office software.
Leica Infinity, a specialised geospatial surveying office software, has been analysed in this work. Infinity is focused on workflows, supporting among the others construction validation and layout tasks. It is designed to easily process, combine, and integrate data collected from different kind of sensors such as GNSS receivers, Terrestrial Laser Scanners (TLS), Total Stations (TS), Digital Levels and Aerial and Terrestrial photogrammetric devices, enabling working in a common reference frame. BIM data are also supported, so comparisons between the design (as-planned) and the actual state (as-built) can be performed. Being able to process and combine these sensor data, and to easily handle the needed transformations to bring all the data in the same reference frame, it provides the perfect tool to compare construction progress with the planned design. Important for this investigation, Infinity also provides a state-of-the-art photogrammetric engine to process UAV and terrestrial images and create Dense Point Clouds (DPCs), Digital Surface Models (DSMs) and Orthophotos as data deliverables. It is also possible to use default base maps (such as Open Street Maps, ESRI Map, Hexagon Content Program, NGS Topo) and connect to customised Web Map Services, to have a visual awareness of the work done in the field and have a better representation of data location.
Having a single environment for managing data and relating the different tasks is important to reduce potential oversights or errors. The goal for the Infinity product is to bring the different sensor data to a geodetically correct and traceable workflow, working in a common reference frame; it uses data dependencies for simplifying the visualization of the various data to the work.
The effectiveness of the approach of using a single software environment for working in a common reference frame is revealed when combining various data sources utilized in different acquisitions, such as GCP for the UAV flights, or control points used for total station resections, or using total station measurements to locate targets for registration of scanner data. The result also gives full data transparency to each of these steps deriving processed results that remove uncertainty and that can be combined in a single processing workflow, resulting in a perfect tool to monitor site construction progress.
A focus of this paper is to refer and explain the utilisation of imagery-derived results within geodetic workflows, and that this processing is combined and taking advantage of the other sensor data available. To make imagery effective with all standard survey geodetic applications, Infinity takes the approach of bringing the image pose position acquired with the UAV GNSS sensor directly to the project coordinate system. The fact that the image poses are already projected in the coordinate reference of the project data removes the uncertainty of projection distortion of the image pose position. When importing UAV data using the WGS84 position, the user is already able to view the location of images with respect to all project data including total station, RTK GNSS data and even more practical for construction, to view the flights along with the BIM model ( Figure 2). One important advantage to this is knowing before processing the images, the data is meeting the task objectives, minimizing possible processing problems. For geodetic confidence, this approach allows the resulting point cloud to reflect a quality directly with the bundle adjustment of the image block. This method has also been patented (Bach et al., 2021). Each point of the point cloud has a confidence value as other data source, such as a total station and GNSS observation, and working with all data sources in a project and being able to visualize the relation to the source of the data is removing the uncertainties for trusting the results. Further advantages to bringing imaging workflows into the common project reference frame is giving the construction site monitoring the ability to combine many imaging events over various periods of time, all within the same project. The utilising of total station or GNSS data for control points for image processing, simplifies site work to quickly address and continue project work. This also makes the workflows more efficient for the various teams who can work with more data to better accomplish these tasks.
Similar as imaging, scan registration workflows are sharing the same ability to work in a common working project. When in a single reference frame, total stations or GNSS RTK data or even points from images-derived point clouds, with a derived confidence value expressing quality, can be applied to targets for scanner data registration. Having all these measurement sources in the same project provides resourceful advantages to keep site work efficient.
When the construction of the structures can begin, utilising data from the BIM models to prepare for the different tasks is helped by breaking down the data and preparing information for the field devices also with Infinity. Providing data for total stations and RTK GNSS devices lets the sensors be utilized for their ability to guide and inform as the site requires. Depending on those tasks and in relation to the job site, robotic total stations resect with the site control and can be used for tracking targets placed on structure elements needing installation and guidance. The measured data can be then visualised with the BIM model ( Figure 3). Being able to view the different data sources together also adds context to location and data validation.

RESULTS AND DISCUSSIONS
The advantages of an integrated data acquisition, and in particular the contribution of UAV data in construction site monitoring, also considering accuracy and completeness, are assessed. To do so, we used as test site one of the largest construction projects in Switzerland recently completed, known as "The Circle", a large-scale business development site at Zürich airport, with many structures and as well underground levels including park house and underground utility facilities. For this project work, various sensors are utilized and contribute to the different phases form start to end, from establishing the project site control network to as-built validation of structures. The construction required the use of many measurement sources including scans by TLS and Multi Stations, discrete points acquired both with Total Stations and GNSS receivers, and images collected with a UAV. Utilizing BIM, the entire job site was available in IFC format and used as a reference (the asplanned model).
For The Circle project, combining scanner data to work with the BIM model required a registration step using targets. To bring the data in the project reference frame, accurate positions for the targets were measured using total stations and connected to GNSS observations. To evaluate verification of construction components, 6 scan setups from a Leica RTC360 scanner were used. The primary acquisition purpose was the validation of the as-built construction compared to design model, with the advantage of using scanners for quickly acquiring data that can be used between different groups of project teams. The data set was also used to verify the structure was within design acceptance about site safety regulations such as inventory of fire exit signs and water sprinklers. The scan data was established with several black and white targets, that were used to bring the registered point cloud to the project coordinate system. The registration was accomplished using the control points in the project coordinate derived from GNSS observations; the total station resected the GNSS observations to position the interior black and white targets and carried those throughout the building structure.
Once the scan data was available, the total station data was used to assign control to the black and white targets. The RTC360 point clouds are normally already stored in the field acquisition in a relative registration. In infinity the step to apply control and optimize the scan group using the total station control points was transparent in the final registration, with the results of the scan setup bundle adjustment to reflect the total station established control. Another example of using different sensors and their data together supporting measure tasks, a Multi Station (MS) was used in several roles during construction. The MS is a precise total station that also provides scanning capabilities. For the installation of window façade elements, the MS was utilized for tracking the façade element to location, and then used to scan the core mounting points to verify installation was within the tolerance. The MS was also used to derive the interior control network that was used for the scanning tasks. For the registration steps, using data directly from a MS station setup including the measurements to targets was possible, simplifying the alignment of scans inside the structures.

The value of imagery in geodetic workflows
For this evaluation, 900 images at two different flight heights were acquired with a Leica AX20 UAV. Images were collected at two heights (60 and 80 m) to arrive at a more rigid photogrammetric block given the complexity of the area. 3D points measured with GNSS receivers were used as Ground Control Points (GCPs) and Check Points (CPs), to check intrinsic precision and global accuracy of the generated point cloud. 3D points have been surveyed taking great care of their distribution, both in planimetry and altimetry, being this a variable that influences the geometric quality and the accuracy of the 3D reconstruction (Ulvi, 2021;Tonkin et al., 2016;Sanz-Ablanedo et al., 2018;Agüera-Vega et al., 2018). Working with some ground constraints due to in progress construction work, points have been measured to homogeneously cover the surveyed area. For this analysis, 9 GCPs and 4 CPs have been used with the distribution shown in Figure 4. 3D points have been also homogeneously distributed as regards to the height component as well, to assure a good vertical reconstruction and avoid scale issues due to wrong camera calibration. Heights of CPs spread from about 429 m to 447 m, this means they are inside the height's range of GCPs, which are measured from about 428 m to 449 m (Table 1 And Table 2). For all 3D points, 3D quality is also available. This is computed as a confidence level during acquisition. These values are used in Infinity as input during image processing, removing the need of manually setting a roughly estimated accuracy value.  Image orientation (including camera self-calibration) is computed in Infinity, based on Structure from Motion techniques integrated with bundle block adjustment, to derive DPC, DSM, Orthophotos, and to compute volume variation by differencing DSM. Measurements on such generated objects can be easily carried out in the set reference frame, such as measuring distances on the Orthophotos as shown in Figure 5. Starting from given initial values, interior orientation has been estimated using GCPs as ground constraint. The accuracy of the images' orientation is computed obtaining RMSE values on CPs for Δ3D of 2.3 cm and for mean reprojection error of 1 pixel. On the other hand, RMSE on GCPs for Δ3D are of 1.1 cm and for mean reprojection error below 1 pixel.
Once the accuracy of the estimated bundle is validated, the DPC, the DSM and the Orthophoto are generated. The point cloud derived from UAV images is supported by scans from a terrestrial laser scanner in those areas where it faces challenges in the reconstruction. Point clouds registration is also carried out in Leica Infinity making use of targets in a cloud-to-cloud registration. BIM, scans, image-derived point cloud, GNSS and TPS points, all fit together and complete each other contributing to the overview of the construction site ( Figure 6).

Figure 6.
Overview of the available data.
Overall, a high completeness reconstruction level is reached. Some areas, though, show that there is still room for improvement ( Figure 7). For such cases, a terrestrial imaging tool such as the Leica GS18 I GNSS RTK rover could contribute to the construction site monitoring, being able to acquire GNSS RTK points and images at the same time, adding details in areas obstructed from a bird's eye view. Results prove that a surveying data fusion software is an important component in supporting a construction site monitoring, also integrating UAV image acquisitions as an effective complementary tool. In fact, UAVs contribute to construction progress monitoring providing accurate photogrammetric-derived point clouds able to cover areas that cannot be easily surveyed with other instruments. Such DPCs can be merged to traditional scans using discrete points and compared to BIM data to check the progress in areas where required accuracy is compliant with the results achievable with a photogrammetric image processing (e.g., sidewalks, roads). For other parts where millimeter accuracy is required (e.g., mounting of Façade plates), scans from Multi Station are more suitable acquisitions. It's then clear that every instrument plays a key role in surveying a complex construction site, and that merging of data offers achievement of completeness. Using a different software for each data source might be a solution for standard surveying; when it comes to complex construction site monitoring though, having a single environment to process, review and interpret the accuracy from all data sources has an important advantage to understanding the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France specifics for a construction phase, as well bringing time and money savings.

CONCLUSIONS
This work deals with the importance of properly working with the large variety of data that nowadays can be more and more easily acquired. It explores the gains of using different data sources to efficiently and fully monitor a complex construction site, where having data captured from different points of view and giving different levels of details is effective to the project's goals, contributing to the different construction phases from start to end, and to the needed overview of the work's progress.
As test site, one of the largest construction projects in Switzerland recently completed, known as "The Circle", is investigated: this is a large-scale business development site at Zürich airport. The project required the use of many measurement sources including scans by TLS and Multi Stations, discrete points acquired both with Total Stations and GNSS receivers and images collected with UAV, each of them adding layers of information and details. Utilizing BIM, the entire job site was available with IFC format and used as a reference (the as-planned model).
The advantages of using an integrated and geodetically relevant software environment, when a high level of data integration is required as in the above-mentioned case, are analysed. Leica Infinity, a specialised software able to merge various data sources for working in a common reference frame, is investigated. Being able to bring different sensor data to a geodetically correct and traceable workflow, working in a common reference frame, makes it effective and efficient for monitoring a complex construction site, also giving full data transparency to the derived results. Infinity combines the data that can be utilized during processing steps, with the goal to achieve quicker confidence to the results. In this document it is specifically shown the value of bringing image data into the project reference frame before performing any photogrammetric processing. In Infinity, images are in fact brought together with all the other source data into the reference frame of the project, and images' poses are transformed upon import to facilitate and make the whole process solution geodetically correct. In this way there is the possibility to visualise at once all the data belonging to a specific site (images with the BIM model as well with other measure data), giving more awareness to the user, and minimizing the sources of errors that might come from the user interaction and the computation. This brings effectiveness and higher global accuracy to the deliverables. As regards point cloud completeness, some areas, where the UAV could not get the level of details and the coverage required, were not completely reconstructed as in the driveway shown in Figure 7. For such cases, a terrestrial imaging tool could contribute to the construction site monitoring, being able to add details in areas obstructed from a bird's eye view. A tool as Leica GS18 I GNSS RTK rover, able to acquire GNSS data and images at the same time, would well complement the already used devices and enabling point cloud generation on obstructed areas, allowing RTK points and images acquisitions using a single tool. Point clouds could be then generated in the office with Infinity.