DIGITISATION IN ROAD CONSTRUCTION: AUTOMATION OF AS-BUILT MODELS

The construction industry is still one of the least digitised sectors worldwide and digitisation in the road construction sector in particular is even more dire. To improve the efficiency of the sector, it is of the utmost importance that construction monitoring tasks, i.e. progress, quality and quantity analyses, are automated. The goal of the presented framework is the automation of the following three vital interconnected subjects: (1) The systematic capture of remote sensing data with low-cost UAVs, (2) the information extraction from survey data and (3) the interlinking and exchange of interoperable BIM data (design, survey and extracted information) using Linked Data technologies. In this paper, the needed data for proper road construction digitisation and the desired data structure is discussed. Additionally, the automation of Real Time Kinematic (RTK)-flight processing is presented. The feasibility of the proposed workflow is examined based on real-life test cases. As a consequence of automation, the homogeneity of data across projects will increase. This will form the basis for further automation in road construction.


INTRODUCTION
The construction industry and in particular the road construction industry, has had very limited digitization (Agarwal et al., 2018).
A recent study in Belgium shows that Building Information Modelling (BIM) is increasingly adopted in the construction sector. In 2021, nearly 70% of the contractors used BIM. This is still quite limited in road construction, where the use of BIM is limited to only 27 % of the contractors (Uzunbacak et al., 2020). Due to the high probability of failure costs, there is a high demand for digitization in the form of BIM usage with quality, quantity and progress analyses (Love et al., 2018). As the research is conducted in Flanders, the specifications and rules of Agentschap Wegen & Verkeer (AWV) are followed (Agentschap Wegen & Verkeer, 2020a). The failure costs combined with the ambition of AWV to apply BIM during the entire lifecycle of the road infrastructure, from design to management, requires a thorough and concise implementation in conventional road construction workflows. In addition to applying BIM, AWV has also introduced Linked Data principles i.e. AWV Object Types Library (OTL) to improve data management. The AWV-OTL is the basis of the new BIM implementation that focuses on asset management. The AWV-OTL is a collection of linked data (LD) ontologies and defines concepts such as object types, properties, relationships and data types using stable identifiers, human readable definitions and formalised constraints. Flemish road designers and contractors are required to deliver BIM datasets to AWV according to the concepts in the AWV-OTL. Using this BIM with the corresponding AWV-OTL is part of the research. Although there are several types of BIM throughout the life cycle of a road construction. These different BIM models will be adjusted and compared based on accurate measurements.
The presented framework consists of three major blocks: (1) accurate surveying of the site, (2) automatic extraction of knowledge from the captured data and (3) the Linked Data to link it all together. These three steps are visually presented in the figure above (Fig1). First, the road construction site has to be captured accurately. Currently, measurements are made with a Global Navigation Satellite Sensor (GNSS) or by using Total Stations (TS) (Tucci et al., 2019). However, these are time-consuming and select measurement methods. The demand for aselect remote sensing technology is therefore high. An ideal method is to utilise a Real Time kinematic (RTK) drone with an RGB-camera attached. This forms a relatively inexpensive and accurate method to fully capture large sites. By using this technology, multiple georeferenced outputs can be generated such as point clouds, meshes, orthophotos and digital elevation models (DEM). An alternative and more robust method is a RTK-drone equipped with a Light Detection And Ranging (Lidar) module. This is more resistant to weather and varying lighting conditions. In addition to Lidar, there is also the possibility to mount a multispectral camera on the RTK drone. This is also more weather resistant and can even be used to detect materials and defects in road construction (Mei et al., 2014). Both latter systems are more expensive compared to the standard RGBcamera. In fact most construction errors can be observed within the visible spectrum as well as the information required for the various analyses, that's why a RGB-camera is used in this paper.
Second, the information must be extracted from the captured data. This process is still a manual process, which is time consuming and error prone. During this procedure, components are visually identified and compared to existing data. These existing data are already identified components from, on the one hand, previous surveying or on the other hand from BIM models. By being able to detect these objects automatically, time and manpower will be saved. Additionally, the automation of detection will also make it possible to detect changes. This is possible because the recognized elements can be compared with each other on two different times and thus also on two different measurements. Process automation enables stakeholders to obtain knowledge almost effortlessly. This object recognition will allow three analyses to be done and automated i.e. quality, quantity, and progress analyses. In the quality analysis the detected objects from a recording can be compared with the planned objects from the as-designed BIM model. In the quantity analysis, the objects can be compared between the recordings to see the difference between two phases. But also compared to the plans to see how much still needs to be done in terms of quantities. And finally the progress analysis can be done by comparing the recognized objects between the different recordings as well as with the plan to see if something is completely finished.
Third, the interchangeability of data among the wide variety of software applications used by all the stakeholders, must be ensured to prevent the loss of information. The greatest benefits of BIM occur when all project stakeholders work together efficiently throughout the course of a project, from early design to completion, by exchanging structured data. In line with the most recent research on data exchange in the construction industry (Rasmussen et al., 2018;Zhang et al., 2018) and the AWV-OTL initiative, this research project explores how to implement a BIM environment between stakeholders using Linked Data for the asset management phase (i.e. AWV), i.e. during the design and construction phase of a road infrastructure project. As a result, stakeholders are able to define, link, exchange and validate content of datasets between otherwise largely disconnected software environments. Resulting in a need for structured data to both ensure automation and implement of Linked Data based on the OTL.
The goal of the entire research is to digitize and automate road construction in general. And that in a way that stakeholders can communicate easily through a homogeneity in data and where linked data can be implemented in road construction. In this paper the automation of georeferenced output from RTK-drone flights and the organised data structure is discussed. In summary, the main contributions of this paper are: 1. The unsupervised workflow to generate georeferenced photogrammetric outputs from RTK-UAV data. 2. A proof of concept for the linked data in combination with BIM for the different analyses e.g. quality, quantity and progress analyses is.

Figure 2.
Methodology for road construction monitoring. An overview of the Linked Data and BIM data structure for the estimation of the quality, quantity and progress analysis.
3. A comprehensive related work on Linked Data usage and the use of UAV data in the road construction industry. The remainder of this work is structured as follows. In section 2 the related work and the background is presented. The methodology is presented in section 3. In section 4 the experiments are discussed. The conclusions can be found in the last section, section 5.

RELATED WORK
Similar to the rest of the structure of this research, the related work is also discussed in 3 major parts. The first part is about the accuracies of an RTK drone and the possible use of GCP's. The second part is about the automated detection of objects from the georeferenced output of the drone. And finally, the last part is about the linked data needed in the research, with a specific focus on the universal language developed by AWV, i. e. the OTL.

Accuracy of the RTK drone and the use of GCP's
Several studies on the accuracy of drones have been conducted, since this research uses an RTK drone only studies on the accuracy of RTK-drone is discussed. In the work of (Štroner et al., 2020) the accuracy of the RTK-drone is examined. The accuracy between a drone flight based on ground control points (GCP's), a drone flight with an RTK-drone without GCP's and finally the combination of the last two, is tested. These experiments indicate that the predetermined accuracies can be achieved with the RTK-drone without the use of GCP's, but they do indicate to be cautious with the z-coordinates when GCP's are not used. In addition to this research, experiments have also been conducted in the investigation of (Taddia et al., 2019) regarding the accuracy of RTK drones. This paper states that the best results are obtained when using one GCP, for nadir flown flights, for oblique flown flights no GCP is needed to achieve good accuracy. The conclusion is that it is not necessary to use GCP's, but it is recommended if possible.

Automatic object detection
There are several georeferenced outputs that may be generated from a photogrammetric project and this will have an impact on how objects may be detected. Not all objects can be generated equally easily from the same output. As a result, different methods will be looked at depending on the objects to be detected. Some different elements of road construction will be discussed for object recognition, as an example, others will be looked at in the remainder of the full research, so will not be part of this paper.
Because curbs are one of the most important objects in road construction, because everything is connected to them and these are constructed early on, it is important to detect them. In the work of (Truong-Hong et al., 2019) point clouds are used to detect curbs. Their method uses is a cell-based method existing of three main steps: (1) filtering ground points, (2) extracting cells containing road curb points, and (3) remove incorrect road curb segments. The conclusion of this research is that it is possible to detect road curbs on point clouds from Aerial Laser Scanning ALS.
In the work of (Commandre et al., 2017) Very High Resolution (VHR) aerial and remotely sensed images are used to detect manhole covers. The paper used a Convolutional Neutral Network (CNN) an improved version of the existent network ALexNet CNN (Krizhevsky et al., 2012) to improve the detection of manhole covers from RGB camera's. The concept is to use two different ways on the one hand geometric filter to detect round objects and on the other hand a machine-learning Support Vector Machine (SVM) based approach. The conclusion of this paper is that it is possible to detect manhole covers from VHR aerial data which is similar to a georeferenced orthophoto.

Linked data, AWV-OTL
AWV has developed OTL with the aim of defining and standardizing the information needs related to infrastructure objects using a semantic model made available as an open standard. Each object type has an unambiguous definition, a number of fixed properties and possible relations with other object types (Agentschap Wegen & verkeer, 2020). A legend book has been prepared that is conform to AWV's OTL and is also compatible with the Grootschalig ReferentieBestand (GRB), an output that was requested before the introduction of the OTL. The GRB is a geographic information system that serves as a topographic reference for Flanders. Because of the compatibility of GRB in the latest version of this legend book, no further consideration is given to the GRB itself. This legend book (Agentschap Wegen & Verkeer, 2020a) describes how to name elements so that they are OTL compatible. Every naming and therefore every element has a dictated quality in planimetry and altimetry. The quality is an imposed accuracy associated with the name and therefore the object. This is element dependent and not project dependent quality. Next to the name of the object there must be an AWV-OTL conform property added with an (Uniform Resource Identifier) URI. This URI is linked with elements from the OTL and the "Standaard Bestek 250". The "Standaard Bestek 250" is a specification imposed for the design of the roads, but is also used for quantities and price calculation. Therefore quantity take offs can be calculated automatically because of the used URI. In addition to this legend book and the object-dependent specifications, there are also project-dependent specifications and these can be found in the BIM-Execution plan (Agentschap Wegen & Verkeer, 2020b). These specifications are focussed on the BIM-side of the cooperation. The main elements contained in this publication are the level of geometry (LOG), level of development (LOD) and the level of information (LOI) which are project specific. Currently, this is project specific because it is a new feature, it is assumed that in the future, there will be a move to a maximum level of detail.

METHODOLOGY
The methodology can be split into two major parts. On the one hand the methodology that deals with the data exchange needed to ensure homogeneity across projects, which provides the basis for automation. On the other hand the methodology to automatically generate georeferenced outputs from RTK drone surveying is discussed. The complete methodology of the data structure can be found in the schematic of Figure 2.

Data exchange to ensure homogeneity
In the first major part of the methodology, the data structure is discussed in order to guarantee the homogeneity of the data. The input to the research is a BIM model in the execution phase (EP), the first green box. There are some specifications that this model must fulfil. In Flanders, the LOD, LOG and LOI must be taken into consideration. Additionally "standard bestek" 250, 260 and 270 must also be taken into account. As a result, following assumptions for LOD LOG and LOI are made: in the BIM-EP the LOD should be 350 and the LOG/LOI should be 3, these two specifications are compatible with one another. Throughout the construction of the infrastructure the BIM-EP model must be periodically adjusted so that at the end of the infrastructure works it can be updated as an As-Built BIM (AB-BIM) model. These periodic adaptations must be automatically done based on the georeferenced output from the RTK-UAV that will be discussed in the second part of the methodology. Depending on the type of adjustments to be made, these can be detected from various georeferenced outputs. In the model, shown in Figure 2, there is a proposal of the various elements that can be generated from the different outputs. The various elements seen in Figure 2 are a thoughtful suggestion and further experiments are conducted to enhance the proposed workflow. When the construction site is finished and the adjustments on the BIM-EP are accomplished, the As-Built BIM can be delivered as output, the red box in the model. Again, the AB-BIM model must take into account the same specifications as the BIM-EP. However, there is an extension of LOG, LOI and LOD for the AB-BIM. Both models will be connected to each other and can inherit things through the linked data called OTL-AWV.

Generate georeferenced outputs automatically from RTK-drone
In the second major part of the methodology, the automation of the georeferenced output from photographs is discussed. In order for the georeferenced output to be compatible with the used plans, the outputs must have the same coordinate system, this depends on the input from the BIM models. As already mentioned, the project is situated in Belgium so the Lambert projections will be used. Because in Belgium several types of coordinate systems are momentarily used, the developed method must be compatible with these different coordinate systems. First, several photos are captured with the RTK drone that is combined with the base station. The amount of photos depends on the velocity of the drone, altitude of the flight and the surface of the construction site, but a minimum overlap of…60% should always be ensured. These photographs are already georeferenced in WGS84 due to the RTK-module that is present on the drone in combination with the base station that is placed on the construction site. Second, because the used processing program, Metashape (Agisoft Metashape version 1.5, 2019), is not capable of generating the necessary coordinate-systems, an implementation must be added. The Exchangeable image file format (Exif) data from the captured pictures must be transformed into the desired coordinate system, by using the implementation. The original Exif data is replaced in Metashape by the coordinates calculated by the external method. Third, the photos with the new Exif data are aligned and different georeferenced outputs are created, i.e. a mesh model, a point cloud, an orthophoto and a DEM model. Fourth, all the different created outputs are stored in a organised folder structure to form the base of subsequent steps in the automated process for road construction monitoring. Last, an RDF-file (e.g. Turlte or JSONLD) is stored with various meta-data from the pictures and of the processing of the georeferenced outputs. This is in order to be able to implement the linked data in the process later. All the previous steps are done in an automated manner, from loading the photos with the RTK-GNSS Exif data to storing the georeferenced output in an organised folder structure.

EXPERIMENTS
The experiments are categorised in two major parts, similar to the rest of the paper. The first part of the experiments discusses the accuracy of the output from the RTK drone which is important for the rest of the research. The second part discusses analyses approaches in order to automate these actions in the future with a goal of transforming the EP-BIM to the AB-BIM.

Experiments concerning the RTK-drone
In addition to the researches presented in related works, several experiments are conducted to test the need of GCPs for accurate georeferenced outputs. First, in order to test the accuracy of the RTK drone some experiments on markers must be performed such as testing different materials and sizes of GCPs for various factors including durability and detectability. The durability of the GCPs have multiple advantages, it is less time consuming because it is possible to capture them once and use them during the entire execution phase. Also, when the GCPs are durable they can be measured on different times to improve the accuracy of the GCPs through the redundancy of the measurements. When different GCPs are used throughout the execution phase, the inaccuracy of the GNSS sensor negatively impacts the outputs of the drone photogrammetry. There are several target types that can be used. In the experiments, printed targets, laminated targets, laser cut targets and 3d printed targets are examined. The experiment went as follows. The different markers were placed on the ground in an open parking space in Kluizen, Belgium. Multiple pictures were taken on different altitudes in different weather conditions in order to test the maximum altitude where the targets were detectable in different weather conditions. The targets were kept outside to test their durability. The tests indicate that printed targets are not durable but quick to make and they have a good detectability. Laminated targets can also be quickly made and are relatively durable, but the sun reflects on them so they are often undetectable. laser-cut targets into wood are durable after some wood treatment, but due to the less obvious difference in colour they are not detectable at higher altitudes. The 3D printed targets eventually gave the best result in terms of durability and detectability, while their production time is only a bit longer.
A second experiments evaluates the need for ground control point if RTK is present. Several automatically detectable targets were permanently attached to the test site. The test site was a soil storage facility situated in Kluizen Flanders Belgium. The targets were measured by different GNSS antennas at different times and on different days. This is to make the measurements as independent as possible and thus reduce the error on the GNSS antenna. Additionally, several flights were made with the RTK Figure 3. Location of the targets on the test site.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-5/W1-2022 Measurement, Visualisation andProcessing in BIM for Design andConstruction Management II, 7-8 Feb. 2022, Prague, Czech Republic drone using a base station, also at different times and also on different days, independently from the GNSS measurements. The targets were automatically detected from the drone flights by the Metashape software and the different measurements of the GNSS-antenna and the drone were compared to each other. The locations of the targets can be seen in Figure 3. For reference, the distance from T1 to T5 is 63.82 meters. The values were rigorously examined and outliers were removed. Because the open-space multipath is minimal, the biggest cause of error was the positioning and number of satellites when the measurement were carried out. Then the averages of the different GNSS measurements were compared with the averages of the RTK drone measurements. The differences between both datasets is presented in Table 1. Both the X, Y and the Z coordinates have accuracies that are comparable to GNSS accuracies. The conclusion of this experiment is that control points are not necessary when using an RTK drone as they do not improve the overall accuracy. This has several advantages such as no GCPs need to be materialized on site. Additionally, they also do not need to be measured which accelerates the time of capturing the site.

Experiment with BIM in combination with Linked Data
In Belgium, BIM adoption is limited in the road construction industry. However, to automate quality, quantity and progress analysis, a standardized input is needed. As such BIM models must be created of every site. Additionally, this 3D model must be compatible with the language AWV-OTL and ideally in a Linked Data way so different stakeholders can easily communicate. First, the initial plans of a site were used to create the EP BIM as discussed in section 3.1 (Fig.4). Second, the drone footage and SfM process metadata were stored using Linked Data so it would be compatible with the AWV-OTL at later stages. To this end, a script was written that extracts the EXIF data from the images and stores in an RDF graph serialised in a .tll file. Similarly, the SfM metadata is stored as an RDF graph serialised as an xml file which. A second script was written that extracts the BIM geometries from the EP-BIM and creates an RDF graph that will serve as the reference for the analysis. Concretely, both the SfM and the BIM geometry graphs will be consumed by the analysis methods that will then in turn output RDF graphs with the analysis results that will be used to generate the AB-BIM.

Experiments concerning object detection on the georeferenced output
In this part of the experiments, several frequently occurring object type instances that can be found in road construction will be manually detected in order to see if the proposed method shows promise. These experiments have the purpose to establish an automatic detection method that will be developed in a later stage. The test site where the object type isntances are manually detected is situated in Mariakerke Flanders Belgium. Because the test site is not completed, the possible objects to be detected are limited. Figure 5 shows the EP-BIM model that was manually created from the heterogenous 2D plans. The three objecttypes that are reviewed, are (1) sewers, where the hypothesis was to detect them form georeferenced photographs, (2) sub-foundation, where the hypothesis was to detect them from the mesh and (3) manhole covers, where the hypothesis was to detect them from the orthophoto. These manual detected objects are compared to the EP BIM model objects. If the error between both are within tolerance, imposed by AWV and different for each object, the AB BIM model can then remain the same as the EP BIM for that object. The accuracies for these specific objecttypes can be found in table 2. When the errors exceed tolerance considering the SfM errors, the AB BIM model will be altered according to the output from the RTK drone.   Table 2. the minimum accuracy to be achieved according to AWV.

Manual detection of sewers
Due to the cylindrical shape, of sewer objects, the hypothesis was that they would be best detected on georeferenced images. This is because these shapes are very error prone in point clouds and meshes. By using an RTK drone, the images that are made are immediately and correctly georeferenced. When the sewers are detected, they will be compared to the as-designed BIM model. Each object has a certain accuracy, both planimetric and altimetric, which is mentioned in a table published by AWV. In the case of the sewers it is 9.8 cm in planimetry and none for the altimetry (Table 2). However, it is still important that the slope of the sewer pipes complies with the regulations, so this must be taken into account. The visualisation of these sewers from the georeferenced picture compared with the BIM can be seen in figure 6. A close-up from one of these connections of the sewers can be seen in figure 7. There are 3 different distances between the detected object and the object from the EP BIM. The axis line from the detected and the designed sewer are compared with each other. The different distances can be seen in table 3 and clearly the second distance exceed the minimum accuracy which means that sewer itself must be replaced in the AB BIM. At the end of the sewer an elbow is placed which causes the difference between the first and the second distance and therefore the second distance exceed the minimum accuracy. The distances are just 2d distances because sewers don't have accuracies in the altimetry. The hypotheses is therefore correct and undistorted georeferenced photographs can be used to detect sewers.   Table 3. the required accuracy in comparison with the measured accuracy for a sewer.

Manual detection of Sub-foundation
The hypothesis is that sub-foundation are best detectable on a mesh, because it is a large sloping surface and would show up better on a mesh than a point cloud. Analogous to the way the sewers were detected, the sub-foundation will also be compared with the EP BIM model. In the case of the sub-foundation the accuracy is 9.8 cm in planimetry and for the altimetry 3.68 cm (table 2). The minimum prescribed thickness, presented in other design guidelines, must also be taken into account. The case detected from the mesh compared to the As-built BIM model is shown in Figure 8. A close-up of the mesh can be seen in figure  9, where it is visible that the border of the sub-foundation isn't detectable in the mesh. The sub-foundation is not finished yet so no conclusions can be made here nor for the planimetry nor for the altimetry. The hypotheses is therefore not answered and a new experiment must be done when the sub-foundation is finished. Figure 8. Detection of sub-foundation out of a mesh compared to the as -design BIM. The sub-foundation is not finished. Figure 9. The border of the sub-foundation isn't detectable on the mesh, the sub-foundation isn't finished yet.

Manual detection of manhole covers
The third and final objecttype are manhole covers. The hypothesis here is that they are best detected on orthophotos. In this construction site there are two sorts of manhole covers, i.e.. Finished and unfinished manhole covers( Figure 11). In figure 12 a close up from the three manhole covers in the model can be observed. Because of the circular shape of the manhole covers, it seems ideal to detect them on an orthophoto. However, the altimetry cannot be directly extracted from an orthophotograph and thus the mesh is also evaluated. The required accuracy of the manhole cover is relatively high, namely 1.72 cm in planimetry and 1.23 cm in altimetry ( table 2).
As already mentioned there are finished and unfinished manhole covers. Only checking the finished manhole covers, is the easiest method. However, there is significant incentive to check both finished and unfinished manhole covers, as this allows assessments to be made more quickly about the manhole covers. The earlier assessments are made in the construction timeline, the lower the failure costs are. In figure 10 it can be seen that the hole of the pit isn't centric with the pit itself. The hole also has another diameter than the manhole cover when it is finished. To control the planimetry of the manhole cover the centroid of the hole can be compared to the centroid of the manhole cover from the EP BIM. However, no conclusion can be drawn in terms of altimetry. For the finished manhole covers it is a lot easier because the diameter of the manhole cover on the orthophotograph should be exactly the same otherwise the method with the centroids of the manhole covers can be used there. Because the hole in the middle manhole cover is blocked there can be no conclusions about this one. In Table 4 the measured accuracies are compared with the required accuracies and the conclusion is that both manhole covers, top and bottom, must be replaced in the AB BIM. The hypotheses was to detect the manhole covers in the orthophoto, it is possible for the planimetric accuracy but the mesh is needed for the altimetry. It could also be done with the mesh only but because of the specific shape from a top view the manhole covers are better detectable on a orthophotograph.   Table 4. The required accuracy in comparison with the measured accuracy for the sub-foundation. The accuracy in altimetry couldn't be checked because the sub-foundation isn't finished. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-5/W1-2022 Measurement, Visualisation and Processing in BIM for Design and Construction Management II, 7-8 Feb. 2022, Prague, Czech Republic

CONCLUSION
The goal of this research is to lay the basis for the linked data in BIM using OTL from AWV. Moreover, some elements are detected manually in order to get those elements automatically from georeferenced output from the drone in later stages.
While the constant transformation from the EP BIM to the AB BIM different analysis are made i.e. quality quantity and progress control. Resulting in the limitation of fail costs because of the frequency of the quality control and in an automatic quantity take-offs in periodic times. As such, various stakeholders can communicate optimally with each other and the manual time and therefore costs are kept as low as possible.
Considering the experiments, it can be stated that GCPs are not necessary when an RTK drone is used in combination with a base station because the achieved accuracy for the georeferenced outputs are the same as the accuracy of the GNSS used to measure any GCP. If GCPs are to be used, 3D printed targets are the ideal solution due to their durability and detectability in different weather conditions.
To update the EP BIM to AB BIM, different objects have to be detected and compared taking into account the accuracies imposed by AWV and other design guidelines. In the experiments, frequently occurring objecttypes were manually analysed given the photogrammetric inputs. Sewers can be detected from georeferenced photographs and manhole covers can be detected from orthophotographs in combination with a mesh. For the sub-foundation a conclusion cannot be made because it was not finished. In the future more experiments on different types of objects.
Finally, the beginning of the linked data pipeline has been provided by serialising the EP BIM and the SfM outputs as RDF graphs. Further automation will be rely on these graphs and will form the core of the data exchange in the project.