SMARTPHONE-BASED REALITY CAPTURE FOR SUBSURFACE UTILITIES: EXPERIENCES FROM WATER UTILITY COMPANIES IN DENMARK

: Inaccurate and inconsistent documentation of subsurface utilities is a reoccurring problem in the construction industry affecting not only the end-users, but all actors involved in designing, constructing, and maintaining pipes, cables and other utilities hidden underground. In this study, a new method for 3D capturing of subsurface utilities, based on a newly developed Smartphone-based Reality Capture (RC) solution is explored. The research was divided into two parts. Firstly a testing of the method accuracy and secondly, an investigation of the usability of the method. The research results firstly showed that the RC solution is a feasible surveying method, that facilitate capturing of as-built utility assets, which can be used as a supporting tool to conventional surveying methods or alone, as the testing showed an accuracy of ±5 cm for the generated point clouds. Secondly the usability testing revealed that the RC solution benefited the utility owners by allowing time-savings on construction projects, as well as generating visual-realistic 3D models of exposed subsurface utilities to be used for quality assurance and planning of future utility work.


INTRODUTION
Lack of accurate and reliable subsurface utility documentation is a real issue in the construction industry making it difficult for planners, excavators, and utility owners to manage underground utility and construction projects. Several research and development projects such as Mapping the Underworld in England (Metje et al., 2007), City Digital in Chicago (UI Labs, 2016) and Digital Underground in Singapore (Van Son et al., 2018) have been and are still investigating how to create more accurate and reliable utility information. The most recent project, Digital Underground, sought to improve the understanding of Singapore's underground infrastructure by developing and implementing a common utility data model (Yan et al., 2021) as well as exploring best-practice methods to capture and generate more reliable underground 3D maps (Van Son et al., 2018) with the goal of creating an underground digital twin (Yan et al., 2019). To achieve this, a case study investigating the use of ground penetrating radar (GPR) equipment, to detect and map buried utility lines was carried out (Van Son et al., 2019). The study showed that the use of GPR has clear advantages compared to conventional surveying techniques, as it is a non-destructive method i.e., no need to remove top surface to locate the buried utilities. The study additionally showed that scanned radar data are laborious to manually process and analyse, which is often not feasible when construction projects are challenged on time and economy -especially on smaller projects. The idea of a city-wide GPR survey approach to 3D map all subsurface utilities was furthermore not recommended. Instead, the study recommended to capitalise on data capturing opportunities of subsurface utilities whenever visible during open excavation using conventional and alternative surveying techniques (Van Son et al., 2019).
Examples of alternative surveying techniques to GPR, are point cloud-based methods, which use data capture from laser scanners or RGB-cameras (photogrammetry) to generate a visual-realistic 3D model of real objects or environments (Van Son et al., 2018). The method is also known as Reality Capture in the construction industry and is used to capture detailed as-built 3D models of built assets. Reality Capture is not yet commonly used for surveying of exposed subsurface utilities in Denmark where conventional surveying techniques are used almost exclusively. Similar tendencies were also observed in Singapore (Van Son et al., 2019).
However, this is not the case for two Danish water utility companies, which are currently exploring the use of a smartphone-based Reality Capture (RC) solution to generate asbuilt point cloud documentation of their water pipe installments during open excavation (Hansen et al., 2020a). The utility companies' primary motivation for testing Reality Capture is to raise the level of documentation quality of their as-built water pipes, in such a way that it can support work processes including quality assurance of as-built utilities, operation and maintenance tasks, and future planning of new construction projects. As of 2021, the two water utility companies have already 3D captured more than 3.500 excavation holes containing water pipes as well as other nearby subsurface infrastructure. This paper thusly investigates the experiences gathered by the utility companies, to develop the use and adoption of Reality Capture as an alternative surveying method for capturing subsurface utility documentation.
The contribution of this paper is divided into three parts. Firstly, in section 2, presenting the workflow behind the RC solution for generating and geolocating as-built point clouds of subsurface utility excavation holes as well as how the utility companies are using conventional surveying. Secondly, an evaluation of the geospatial accuracy of the point clouds generated from water pipes, whereas the methodology behind is presented in section 3. The results are then presented and discussed in section 4. Thirdly, the RC solution is evaluated in terms of usefulness focusing on which added value it facilitates compared to conventional surveying. This was done by conducting and analyzing semistructured interviews, as described in section 3. Challenges and limitations identified in the RC solution and evaluations methods are furthermore described in section 5.

SURVEYING OF SUBSURFACE UTILITIES
This section presents a description of how the conventional surveying and the smartphone-based Reality Capture solution are used in two water utility companies for surveying and documenting as-built water pipes.

Conventional surveying
Water pipes are often placed in open excavations making them visible after installment. The utility companies, therefore, make sure to survey the water pipes when the excavation holes are still open, as it provides them with the the best opportunity to collect accurate information of their built assets.
Equipment used for conventional surveying includes GNSS real time kinematic (RTK) and a total station to measure points along the visible water pipes as shown in figure 1. These tools generate a trajectory of sparse geographically coordinated points, which are afterwards processed into digital vector lines representing the installed water pipes. Informative attribute values are also added to the vector lines, such as depth, thickness, material, time-ofinstallment, etc. The mapped and documented utility lines are then stored and managed in GIS. This is common survey practice for most water utility companies in Denmark, as well as other utility company types, dealing with gas, sewage, electricity, telecom etc. whenever installed during open excavation. The utility companies have their own requirement specifications for surveying of installed water pipes. In terms of horizontal and vertical accuracy, a tolerance of ±5 cm is acceptable. This is easily achievable with survey-grade GNSS-RTK equipment, as this equipment has an accuracy threshold of ±1.5 cm. The accuracy requirements set by the utility companies are also well within the new and forthcoming Danish nation-wide legislation describing accuracy requirements for exchange of utility information between utility owner and excavator (KEFM, 2019). The requirements are ±50 cm in both the horizontal and vertical direction and will be mandatory from medio 2023 for all utility owners surveying newly installed or repaired surface utilities in public areas. However, "soft" cables such as electricity and telecom, are not obligated to document vertical information, having to only provide indicative depth documentation.

Smartphone-based Reality Capture
The smartphone-based Reality Capture (RC) solution used by the two water utility companies was originally co-developed under an innovation partnership between one of the water utility companies and their collaborating contractors and a surveying company. The purpose of the partnership was to co-innovate new work processes and solutions dealing with planning and constructions of underground water pipes. The surveying company within this partnership has further developed the RC solutions since its first prototype was developed two years ago and is now publicly available as a service called SmartSurvey 1 . The two utility companies which participated in the study, have prior to the public release tested the RC solution as an alternatively as-built 3D documentation method. The RC solution has not yet been fully adopted by the companies but has seen a gradually increase in use during the two-year test period.
At its core, the smartphone-based RC solution is a simple and easy-to-use smartphone app/service that is geared towards generating 3D as-built geolocated point clouds of subsurface utilities during open excavation. The RC solution is based on close-range photogrammetry using video recordings captured from a camera-equipped smartphone. Metashape by (Agisoft, 2021) is used for the photogrammetry process and is automatically handled on a server managed by the surveying company. Accurate Ground Control Point (GCP) markings must, however, be marked around the excavation hole and surveyed to geolocate the generated point clouds. A minimum of four GCP markings has currently been found to be sufficient by the developers of the system. It is, furthermore, important that the user in the field capture the markings within the view frame during the video recording to ensure that the geolocation process can be completed.
As part of the RC solution the point clouds are made accessible in a 3D web platform, called PointView 2 , also developed by the surveying company. The 3D web platform uses Potree, a free open-source WebGL-based point cloud renderer (Schuetz, 2016), as its 3D rendering framework. A multi-window view setup of the 3D web platform is shown in figure 2, rendering a point cloud (left 3D view), which location is shown on a map (right 2D view). The whole process from capturing to viewing point clouds follows a semi-automated workflow that is depicted in figure 3. It is noticeable that the user in the field i.e. utility owners or contractors only need to be equipped with a marker and a smartphone to complete their part of the RC process. The process includes three easy steps: (I) creating GCP markings around the excavation hole, for instance with a spray marker, (II) video recording the excavation hole from all angles by circling around while pointing towards the hole, and (III) uploading the video via the RC solution app. The captured image material is then sent to a server managed by the surveying company. After the first step is completed, a surveyor then needs to measure the coordinates of the GCP markings with GNSS-RTK equipment and upload it to the same server. It is noticeable that the surveying of GCP markings can be done without the presence of the user in the field as long as the user has left the markings untouched. This also means that the user, for instance the contractor, can proceed to fill in the excavation hole still leaving the markings untouched after the second step is completed i.e., the video recording. Using photogrammetry and GCP are well-known techniques to generate and geolocate 3D models and is not in itself a novel approach in the industry. However, combining these approaches in a complete, simple and easy-to-use solution designed for documentation of subsurface utilities that requires minimum equipment and actions from the user's perspective is what makes this solution unique.

METHODOLOGY
Evaluation of the smartphone-based Reality Capture (RC) solution was divided into two parts. Firstly, evaluating the geospatial accuracy of the generated point clouds and secondly, interviewing respondents regarding the usability of the RC solution.

Accuracy evaluation
A total of 41 point clouds were selected for an accuracy assessment, measuring the horizontal (XY) and vertical (Z) displacement in 52 Check Points (CPs). The point clouds where selected so it represented a variety of different sizes of excavations ranging from 1-20 m 2 in hole aera. CPs were marked onto the installed water pipes and surveyed with survey-grade GNSS-RTK equipment similar used in conventional surveying with an accuracy of ±1.5 cm. Most of the point clouds contained one CP with some of the larger excavation holes including up to 4 CPs. The CPs were identified in the point clouds, as shown in figure 4, and the extracted XYZ-coordinates were compared to the surveyed CP coordinates through calculating of an error distance represented as the Root Mean Square Error (RMSE). The horizontal (RMSEXY) and the vertical (RMSEZ) accuracy error measurement were calculated as shown in Eqn. 1 and Eqn. 2. The calculations were done following the guidelines of the Geospatial Positioning Accuracy Standard (FGDC, 1998) .

Usability evaluation
Interviews were conducted in two sessions, with a total of four respondents participating. This included employees from the two utility companies and a single respondent from an excavation company.
Each of the respondents were chosen based on her/his involvement and experience using the RC solution. Information about the participating respondents is shown in Table 1.
Respondents 1 and 2 participated in the first interview session and respondents 3 and 4 in the second. The semi-structured interviews were conducted, following the guidelines by Tangaard & Brinkmann (2015). Semi-structured interviews were utilised to provide the interviewer the ability to ask both pre-determined questions as well as potential follow-up questions that could occur during the interview session.
The interviews were sound-recorded and transcribed, in order to facilitate the analysis of the interview data. An initial reading of the transcribed data revealed re-occurring themes, which were then used to structure the main data-analysis. In all two main themes were identified: (I) surveying and excavation workflow and (II) added value of 3D point clouds. It is important to note that the horizontal and vertical accuracy measurements of 3.04 ±1.60 cm and 3.26 ±2.21 cm mean error and standard deviation values, are within the accuracy threshold mandated by the two utility companies, set at ±5 cm. Looking at the upcoming Danish national accuracy requirements, it is also well within the accuracy requirements set at ±50 cm (KEFM, 2019).

Accuracy evaluation
The calculated horizontal (XY) and vertical (Z) accuracy errors are presented in a frequency histogram as shown in figure 5 and figure 6 respectively. Both calculated accuracy error datasets shown in the histograms shared similarities. Noticeably for both are that around 2/3 of the error measurements are within the 0-3 cm range. However, the vertical errors have two larger error outliers within the 8-12 cm error range, compared to the horizontal errors, which is the main reason for RMSEZ being higher than RMSEXY. This is because the accuracy error is calculated by using RMSE, which penalizes large errors more compared to using for instance Mean Absolute Error (MAE) that equally weights all averaged error values. Furthermore, it can be assumed that factors contributing to a large RMS error is related to the accuracy and deviation of the physical measured XYZ coordinates in the (i) GCPs, (ii) CPs and (iii) in the selection of GCPs in the captured image material as well as selection of CPs in the point cloud. This is especially true for measuring Z-coordinates with GNSS-RTK equipment which are well-known to differ under challenging outdoor conditions. Moving forward it is clear, that eliminating those larger errors would help achieving an even more accurate level for the RC solution.

Usability evaluation
Results from the semi-structured interviews evaluating the usability of the RC solution are divided by two themes, (I) surveying and excavation workflow and (II) added value of 3D point clouds. Generally, when questioning the utility companies during the interviews the conversation naturally gravitated towards comparing the RC solution with conventional surveying. Thus, the following presented results and discussions of the interview data will reflect this.

Surveying and excavation workflow:
Some of the respondents noted that the RC solution enabled an improved workflow for the digging contractor by removing most of the waiting time for a scheduled surveyor to survey the exposed asbuilt utilities and thus resulting in time savings for the entire construction project. This was especially pointed out by respondent 2, 3 and 4. Respondent 4 further elaborated from an excavator's perspective, that being able to finish and move onto the next digging site is a huge benefit and something generally new the RC solution offers compared to how a traditional digging and pipe instalment job is carried out. Often contractors need to return to the dig site the following day as a surveyor is rarely available at the time as when the water pipe instalment is ready for surveying.
However, for this improved workflow to function, respondent 4, who has carried out most of the GCP markings and video recordings for utility company B, noted that placing the GCP markings need careful consideration, as they must stay untouched and visible after filling up the excavation hole. This is something The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-4/W4-2021 16th 3D GeoInfo Conference 2021, 11-14 October 2021, New York City, USA that can be considered as a new practice the RC user needs to learn. An example of badly placed GCP marking is shown in figure 7, where a GCP is marked on an iron plate, which would likely be moved during the excavation hole closing process. In discussions about how to improve the workflow further, all respondents mentioned wanting to survey the GCP markings on their own, or give the task to the contractor, to eliminate the need of a land surveyor in the field. It was also discussed if the geolocation process done by measuring GCP markings could be replaced by GPS data retrieved from the same smartphones used for video recording. It was, nonetheless, made clear by respondent 3, that this would, at best, result in a 1-5 m accurate point cloud, which would not be sufficient for documentation purposes. Respondent 1, 2 and 3 however, noted that based on their current needs, the point clouds generated by the RC solution had a satisfying accuracy level. Thus, underscoring that geographic accuracy is key with respect to the generated point clouds as it is the only way to compare it with other geospatial data.
Respondent 3 also brought up the idea of integrating a highaccuracy GNSS-RTK antenna on the smartphone used for the RC solution to do the geolocation process automatically. Thus, entirely skipping the GCP marking and surveying process. This solution was also explored by Geosystems France 3 . However, the overall opinion from the respondents was that they appreciate the simple approach and minimum equipment needed for the RC solution. Even though a more advance setup could potentially be more efficient it might also become more complex, and more equipment dependent. As the setup is now, needing only a smartphone and a spray marker, it is an easy and simple approach that is very accessible and usable in the field.

Added value of 3D point clouds:
The interviews showed that the point clouds benefited the utility companies in a wide range of usages. Respondent 1 described the GIS team's current testing of using the generated point clouds as an input source for creating vector lines of installed water pipes in their GIS utility management system. Having the visual-realistic point clouds available in the 3D web platform furthermore helped to Respondent 2 noted that point clouds could be used as an excellent tool for quality assurance of the agreed as-built instalments. With respect to the same type of inspection process, respondent 2 also mentioned that the point clouds could be used as documentation to prove that other types of closely located utilities (as seen in figure 7) have not been damaged during the excavation and instalment work.
Another usage, expected by the respondents are for future utility projects planned on the same location, where previously captured point clouds are available. They expect that the visual-realistic view of underground infrastructure will make the design and planning process more efficient. It is, furthermore, beneficial on locations where installed water pipes are crossing under other types of utilities (as shown in figure 7), because by the respondent's experience, other utility map records, often lack accuracy and completeness. It was further noted, that this is especially true for "soft" cables such as electricity and tele-com, which is coherent with research by Hansen et al. (2020b).The research, furthermore, showed that using Augmented Reality for visualising as-built point cloud models in the field would be a valuable tool for utility work planning, in order to potentially prevent damage of subsurface utilities during excavation.
On the same note, the respondents also hoped, that other utility owners will start to capture their excavation holes and then exchange 3D capture data with each other. This is an interesting observation, as the continued use of the RC solution or similar solutions could potentially serve as an as-is documentation source for the entire body of underground utility networks, delivering high spatial accuracy and completeness. This falls in line with the Singapore study (Van Son et al., 2019) which suggested that every open excavation is an opportunity to update existing utility records instead of solely relaying on GPR methods. However, the idea of exchanging 3D capture data in Denmark has some challenges. Firstly, other utility owners must start using 3D capturing solutions and secondly that there are no common platform or programme pushed by the Danish geospatial authority that supports or incentivises exchange of such 3D capture data or 3D GIS data in general. Currently the Danish utility information exchange model is undergoing an implementation to use a newly developed GML-based utility data model, which will only support regular 2D GIS data (KEFM, 2019).

CHALLENGES AND LIMITATIONS
During the testing and evaluation of the RC solution some challenges and limitations were observed. These are presented below and will serve as future work to further developing and researching the RC solution.

Challenges
Documenting large and deep excavation holes. Excavation holes for water pipe instalments and repairments are often small compared to holes excavated for sewage projects. General dimension for water pipe excavations is ranging from 1-20 m 2 in area and a depth of 1 m. On sewage projects the excavation area is often considerably larger and deeper, sometimes including an entire road section, thus making 3D capturing challenging as more GCP markings are needed as well as more area to video record at a greater distance.

Messy and incomplete generated point clouds.
A final aspect challenging the RC solution is that the responsibility of capturing image material is given to the users in the field instead of surveying professionals that has expertise in photogrammetry. Even though this challenge is accommodated through illustrative guides and an instant calculated blur and error factor to filter poorly captured video material out, this sometimes has consequences, as shown in figure 8, in which the water pipe appears incomplete and messy. Poorly generated point clouds are likely caused by random and shaking camera movement or lack of angle range capturing the water pipe. In some circumstances, the RC user was met with challenging terrain conditions making it difficult to walk around the excavation hole whilst video recording.

Limitations
The accuracy evaluation does include a lot of parameters such as the selected point clouds for testing has different amounts of GCPs and different sizing in excavation hole area ranging from 1-20 m 2 . A different accuracy testing design is further needed to evaluate the accuracy impact of the numbers of GCP necessary as well as the impact of distance from GCP (located around the dig hole) to CP (located on the water pipes). Thus, resulting in a more wholesome accuracy testing of the RC solution.
The interview data, used to evaluate the usability of the RC solutions was only collected from two of Denmark's largest water utility companies. The study would have benefited from interviewing other utility companies using RC to achieve a better understanding of how point cloud-based capture methods are utilised across smaller-sized utility companies as well as utility companies managing other types of utilities. The generalisability of the study is additionally limited by the small number of respondents.

CONCLUSION
This paper presents a smartphone-based Reality Capture (RC) solution geared towards documenting subsurface utilities in open excavation holes. Through semi-interviews with respondents from companies having utilized RC for two years, it was revealed that the RC solution has an easy and simple workflow and a low equipment dependency, which makes the solution very accessible in the field. The interviews showed that benefits such as, better quality assurance of as-built work and a better-informed decision-making process when planning new excavation work at the same location, can be achieved using the solution.
To compare the accuracy levels of the RC solution with conventional surveying methods, an accuracy assessment was carried out on 52 check points marked on the captured water pipes. It showed a horizontal and vertical accuracy of 3.04 ±1.60 cm and 3.26 ±2.21 cm mean error and standard deviation values, respectively. Which conservatively speaking, can be estimated to an overall accuracy of ±5 cm. Noticeably, this was within the accuracy requirements of ±5 cm, set by the utility companies and therefore acceptable. It was additionally well within the future accuracy threshold of ±50 cm, soon to be implemented in the Danish construction industry.
Overall, the RC solution was concluded as a useful surveying method, which captures unique and comprehensible 3D documentation of as-built utility assets, compared to a traditional outcome of conventional surveying. The evaluation of this study finally showed that the use of RC is feasible solution, which was also reflected through the continued use of the RC solution by the utility companies.
Looking forward, one clear advantage of the RC solution is that it automatically captures other exposed utilities within the same excavation hole. The collected 3D capture data, has the potential to serve as an as-is data source update to reconciliate legacy utility information which is known to lack spatial accuracy and completeness. An investigation on how to approach this is further needed, as the Danish utility data exchange model does not support 3D capture data.

DISCLAIMER
The authors would like to disclose a possible conflict of interest as the second author is also shareholder in the commercialization of the RC solution presented in this paper. We do however not believe that the results have been affected by this.