RPA POSITIONING ERROR INFLUENCE ON CLOSE RANGE PHOTOGRAMMETRY FOR INDUSTRIAL INSPECTION

In oil and gas offshore platforms, special pipelines as flexible risers make the connection between the ocean floor structures and the platform in extreme environmental and operational conditions. Periodic inspections are necessary to assess their integrity. As industrial climbing for inspection is expensive, extremely dangerous and time consuming, qualitative visual inspection with Remotely Piloted Aircraft System (RPAS), also known as drones, are being successfully applied for remote inspection of offshore flares and risers in a much safer, quicker, and cheaper way. These experiences motivate the 3D photogrammetric inspection of risers using RPAS, considering restrictions like layout of the inspected structures and surroundings and inability to prepare the scene. In this paper, taking advantage of the position information provided by the RPAS, the reconstruction and scale of the test scene were made using only GNSS data, GNSS and scale bars, RTK, and RTK and scale bars. Calibrated artifacts were used to evaluate the results and they include a PVC pipe with artificial defects simulating a riser, a pyramidal pattern with four spheres, and scale bars. The results showed that, as expected, the worst results are for GNSS data with error standard deviations of 0.35 mm compared with 0.20 mm or less for other options. For the sphere’s artifact, relative maximum sphere spacing errors are 9.3% for GNSS, 1.9% for RTK and 0.26% using scale bars. In any case it was possible to identify the defects in the pipe with good quality and with much more detail compared with a climbing inspection.


INTRODUCTION
Photogrammetry is a well know technique that can be used to perform a three-dimensional reconstruction of a scene or object from a set of images. Among several important aspects of the photogrammetric procedure, the network design, or the camera stations planning, must be carefully chosen for a proper image acquisition configuration (Remondino and Fraser, 2006). In several applications, due to spatial restrictions, size and height of the scene or object, limited access, and difficulties to achieve proper camera stations, the use of Remotely Piloted Aircraft System (RPAS), also known as drones, is becoming widespread. The growth of RPAS use in visual and photogrammetric 3D inspections is due to several factors, including versatility, speed, ease of use, and reduction of human risks (Nex and Remondino, 2014) (Jordan et al., 2018).
In oil and gas offshore extraction platforms, special pipelines can make the connection between the ocean floor structures and the platform. These especial pipelines, called risers in the portion not touching the ocean floor, are used in extreme environmental and operational conditions. Thus, it is necessary to carry out periodic inspections to assess their integrity as they can suffer damage, such as external abrasion, diameter variations, ripples, deformations, and dents during their operation. These defects can reach meters long and range from a few to hundreds of millimeters depth and are generally detected from external geometry measurements. Traditionally, the external riser inspection between water level and its connection to the platform is performed by an industrial climber, as Figure 1 illustrates. These inspections are done in a hazardous environment. They are expensive, extremely dangerous, and time consuming.
More recently, RPAS with integrated camera for visual inspection are being successfully applied for remote inspection of offshore platform structures like flares (Marinho et al., 2012) and risers in a much safer, quicker, and cheaper way. These experiences led to the development of a research project for the 3D photogrammetric inspection of risers using RPAS for image acquisition in a proper way, considering several restrictions, including environmental conditions, layout of the inspected structures and their surroundings, inability to prepare the scene for measurements with the inclusion of targets and scale bars, and equipment limitations for remote image acquisition. The sensors embedded in RPAS allow their control and positioning/orientation in space. Typically, these sensors include: gyroscopes, accelerometers, barometric altitude sensor, magnetometer, Inertial Measurement Units (IMU), and Global Navigation Satellite System -GNSS (Eschmann et al., 2012).
When an image is acquired, position and orientation information, already compensating for the movement of the camera gimbal, is recorded in a geotag (Peppa et al., 2019). In the photogrammetry reconstruction process, this information can be used as an initial estimate of the position and orientation of images in space and even to define the scale of the scene. However, the GNSS signal has a limited accuracy and can be affected by different sources of error, such as electromagnetic interference, signal shadow regions, atmospheric conditions, and number of detected satellites. The use of only this type of information can result in errors in the order of meters in relation to the actual positioning of the aircraft, which can result in prohibitive scale errors for an adequate inspection. However, there are RPAS equipped with a position correction and orientation technology, called Real Time Kinematics (RTK), which allows the reduction of the position errors to the order of centimeters in ideal conditions (Carbonneau and Dietrich, 2017).
Photogrammetry, in essence, is a dimensionless technique (Luhmann, 2010), so for applications where it is necessary to carry out dimensional industrial inspections, the definition of an absolute scale is typically done using objects that have wellknown dimensions, such as scales bars inserted into the scene. The errors that affect photogrammetric measurements also come from several other sources, such as: network design, camera calibration, camera vibrations and/or scene movement at the time of acquisition, resolution, and quality of the sensor and lens (Sieberth et al., 2015).
In industrial applications, it is common to need to conduct inspections in places which are difficult to reach, where it is often not possible to prepare and insert information on the scene, such as calibrated scales and coded targets. With this, it is necessary to find alternatives to scaling the scene in photogrammetric processing, such as the use of navigation information that is based mainly on GNSS or GNSS + RTK data. Thus, there is a need to perform a metrological analysis evaluating the associated measurement errors when using the various forms of scale definition for photogrammetry.
In the present paper, the influence of image position and orientation data from the RPAS controller, using only GNSS or GNSS+RTK, on the quality of measurement results is performed for the same set of 50 acquired images, in a pre-defined network design. The analysis includes measurements, using a network design developed for offshore platform inspections, of calibrated patterns, scale bars, and a calibrated pipe with synthetic defects like a riser.

PROPOSED RESEARCH AND METHODOLOGY
This paper compares the photogrammetric measurement errors obtained with navigation data with or without the use of RTK. The results are compared with those obtained with scale bars in the scene. For the same set of 50 acquired images, the measurement scale was indirectly defined only by the posture contained in each image, defined in the geotag, or additionally by the scale bars. Thus, four different photogrammetric process strategies are obtained: 1) Only using information from the GNSS; 2) Using information from the GNSS and adding the scale bars information before camera self-calibration optimization and dense cloud calculation; 3) Only using information from the RTK; and 4) Using information from the RTK and adding the scale bars information before camera self-calibration optimization and dense cloud calculation. In all strategies, selfcalibration is performed, as there are no guarantees that a precalibration is stable enough, especially in real measurement conditions in offshore platforms, where it may be necessary to make adjustments to the camera according to the conditions found at the time of the acquisitions. Figure 2 shows the general flow chart of the processes. The measurement scene consists of four scale bars with coded targets, a 300 mm diameter PVC pipe with calibrated synthetic defects similar to a riser, and a pattern with four spheres, as shown in Figure 3. The spheres pattern allows evaluations similar to the VDI/VDE guideline (VDI/VDE 2634, 2002 that presents a methodology for evaluating optical measurement systems, in which it is possible to estimate the systematic Sphere Spacing Error -SSE and the random Probing Error -PE separately. The calibrated pipe allows the direct comparison between reconstructed and reference surfaces.  To obtain good results using photogrammetry, especially when including self-calibration, some important factors mentioned in the literature must be followed. The photogrammetric acquisition network, that is, the position and orientation of the camera in relation to the scene for each acquisition, must be carefully planned with good convergence and baseline to distance ratio, high number of images per 3D point, variety of camera roll angles, and/or an object with a strong 3D distribution. For a better correspondence of points between the images, a percentage of about 80% overlap and acquisitions in sequence is recommended (Nex and Remondino, 2014) (Atkinson, 1996) (Luhmann, 2010).
Considering these factors for a successful and high-quality measurement on one hand and the real conditions found in the offshore riser measurement task and limitations with the RPA and camera gimbal used on the other hand, a trajectory for image acquisition is proposed. The details and previous simulations of this developed trajectory can be found in (Buschinelli et al., 2020).
The developed network design for the RPA trajectory is a fusion of convergent and planar trajectories usually used in close-range industrial photogrammetry (Luhmann, 2010) and in aerial terrestrial mapping (Nex and Remondino, 2014), respectively. Figure 4 shows the network design for the RPA trajectory and image acquisition, called serpentine trajectory. Using this trajectory, a good convergence is achieved around the riser longitudinal axis in the horizontal direction, but the RPA moves in a straight line in vertical direction, along the riser longitudinal axis. For five vertical lines with a Δα=15° step around the riser axis, an approximately 1:1 base to distance ratio with good convergence is achieved. The vertical displacement between images is used to achieve around 80% overlap between successive images. No camera roll angles are used in the image acquisition as the camera gimbal has a limited roll angle (±20°) (DJI, 2019a) and is used only for camera stabilization during RPA flight. In the vertical direction, the image acquisition starts and ends before and after the area of interest in the riser respectively, assuring a high number of images per measured point.

Materials
The equipment and materials used in the experiments are listed below.

2.1.1
The RPAS DJI Matrice 210 RTK v2 is used for industrial applications. It is equipped with multi-frequency GNSS receivers with RTK functionality using a fixed station that allows better stability and precision of position and spatial orientation (DJI, 2019b), being able to perform flights using, or not, the RTK antenna. According to the RPA datasheet (DJI, 2019b), the calculated 3D accuracy using only GNSS is about 2.2 m and using the RTK is 0.2 m. This information is used in the photogrammetric process as weights for the positioning data used in the reconstruction. AoV: angle of view; FoV: field of view, GSD: ground sample distance.

PVC pipe:
A PVC pipe similar to a riser was used. It contains some artifacts and artificial defects on its surface. The PVC tube was calibrated with expanded uncertainty of U=±0.075 mm using a reference fringe projection measurement system ATOS Compact Scan 2M (GOM, 2020). The generated reference CAD files were used as a reference to compare and evaluate the results. These reference files also allowed comparison with artificial defects to assess the resolution capacity of the measurement process. Figure 5 shows the PVC pipe used.

Spheres pyramidal artifact and scale bars:
An artifact composed of a pyramidal structure in structural aluminum profiles and four steel 100 mm diameter spheres with texture application on its surface was used. The component was calibrated using the Faro Platinum measuring arm with a measurement expanded uncertainty of U=±0.06 mm. In the same pattern, four scale bars were set. The scale bars consist of an aluminum structural profile and coded targets in their ends which are automatically recognized by the photogrammetric processing software. Calibration of the distance between the centers of the coded targets in the scale bars was performed with a Renishaw XL-80 interferometric laser and magnifying lens with expanded uncertainty of U=±0.09 mm. The structure obtained can be seen in Figure 6. The distances between the centers of the spheres are listed in Table 2 and the length of the scales in Table 3.   Table 3. Scale bars lengths.

Acquisition and photogrammetric processing
For the images acquisition, a flight with a serpentine trajectory was performed (Figure 4) with a minimum overlap of 80% between the images, with a vertical step of about 0.3 m. The RPA to pipe distance was approximately 5 m, which is considered a minimum safe distance for offshore acquisitions. 3D processing was done using the Agisoft Metashape 1.7.2 Professional Edition software (Agisoft, 2021), with the following configurations: align photos with accuracy high, camera self-calibration and optimization with 8 parameters (f, cx, cy, k1, k2, k3, p1, p2), rolling shutter compensation enabled, build dense cloud with quality high, and build mesh with face count high and disabled interpolation. The images were obtained with the RPA when in hovering mode. At the time of the acquisitions, the average wind speed was of approximately 17 km/h. Figure 7 shows the mesh and the camera stations when images were acquired by the RPAS.

Figure 7.
Serpentine trajectory performed for image acquisition, resulting 3D mesh and camera stations.

GNSS:
The operation with GNSS is performed with the receiver of the aircraft itself, thus making it possible to carry out trilateration between satellites and RPA. From this step it is possible to know the positioning of the system in space (Youssef and Youssef, 2007). In this stage of the experiment, the flight was carried out with the information of position and orientation of the images using only the data provided by the GNSS device (without RTK corrections). The 3D accuracy using only GNSS is of about 2.2 m and is used as weight in the calculation process.

GNSS+RTK:
The flight was carried out with the corrections provided by the RTK system, through differential approaches that model the spatial errors correlated in a regional network and interpolating corrections. The RTK base was fixed at about 10 m from the RPA. After initialization, the user waited 30 minutes for its use. The calculated 3D accuracy using RTK is of about 0.2 m and is used as weight in the calculation process.

Scale bars:
When scale bars were used to define the scene scale, the reconstructions were processed, initially, with the GNSS or GNSS+RTK information. However, before camera self-calibration optimization the targets were detected and the four scale bars B1, B2, B3, and B4 contained in the scene were used to set the scale. Afterwards, the dense cloud and mesh were calculated.

Surface deviation error
The PVC pipe measurement data presented in this paper were evaluated by comparing measured and calibrated surfaces using the GOM Inspect 2019 software (GOM, 2019). The alignment of the result of photogrammetric processing and reference measurement, considered as ground truth (GT), was carried out through least squares adjustment between surface points. The comparison is made by evaluating the point distances between the reconstructed surface and the GT. The comparison is shown as deviations color maps and the distance standard deviation between points.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition)

Static (on ground) RPA position error
With the RPA static positioned on the ground, 80 images were acquired with a 5-second time interval between them. This process was carried out using GNSS and repeated using GNSS+RTK data. This test makes it possible to determine the positioning precision of the RPA for the different options at the time of the tests and to confirm the information from the RPA datasheet.

RESULTS
During flight and image acquisition, the information from the GNSS and RTK was obtained with signal from 10 satellites. The RTK base station was turned on enough time before the flight, about 30 min, to stabilize the corrections sent to the RPA. The next sections present the results obtained.

Camera calibration
The camera's self-calibration and optimization was performed on the Agisoft Metashape 1.7.2 Professional Edition software by extracting features from the scene, with no detection of the targets, using the 50 measurement images for each experiment. This was done to simulate the real offshore acquisitions on nonprepared scenes, except for the evaluations that include the scale definition by scale bars. No camera roll is done, as recommended for self-calibration, due to camera gimbal limited range. Table 4 provides a summary of the results of the self-calibration adjustments for the camera in the processing strategies. The intrinsic parameters are listed as follow: focal length , principal point offset (cx, cy), coefficients of radial and tangential distortions (k1, k2, k3, p1, p2) of the lens together with their estimated errors standard deviation (Brown, 1971). A difference of about 20 pixels in focal length was found between calibrations using, or not, the scale bars, probably due to the use of coded targets detected before camera alignment and self-calibration. Significant variations can be found in principal point offset and a relatively high correlation with focal length (e.g. -0.50 with cy) can indicate a not optimal camera network for self-calibration. Figure 8 presents the SSE for all processing strategies for all pairs of spheres in the sphere's pyramidal artifact. It is possible to identify that the measurements that include the scale bars information have the lowest scale error, as expected. In both cases that use scale bars, the maximum SSE represents 0.26% of the sphere centers distance. When comparing both cases that do not use the scale bars information for scene scale definition, i.e., GNSS and RTK information only, the measurements based on GNSS information has the highest measurement errors. For the GNSS, the maximum SSE represents 5.9% of the sphere centers distance. For the RTK, the maximum SSE represents 1.2% of the sphere centers distance. The same kind of analyses was done with the scale bars in the scene, resulting a maximum relative error of 5.6% for GNSS, 1.6% for RTK and 0.04% for scale bars. Differences from scale bars and SSE errors are due to several factor including the use of a manual selection of point cloud for each sphere and texture-based point cloud calculations. These results are compatible with the contributions from the higher uncertainties in the GNSS data in relation to the uncertainties in the RTK data (Dong-feng et al., 2009). This effect can also be accounted by a relatively poor camera self-calibration, as the desired camera roll angles were not suitable for the camera gimbal used. The advantage of using the SSE to analyze the influence of processing strategies is due to a better separation between the error's components. Thus, in this analysis, it is possible to better remove the random component in the point cloud, as opposite to the surface comparison analyses in the next section.

Surface comparisons
Comparisons were made between the measured surfaces in the PVC pipe and the reference surface (GT), in the region of the pyramidal artifact (50 mm x 45 mm x 15 mm), and artificial defects (100 mm x 300 mm x 3 mm), as can be seen in Figure 4. In this analysis, systematic and random errors act together influencing the quality of the measurement results, as with the analysis of data from actual field inspections. Figure 9 shows the comparisons made between the surfaces. The next evaluation carried out consists of comparing surfaces in the region of artificial defects in the PVC pipe. Figure 10 illustrates the deviation maps analyzed. Through the comparison between surfaces for the pyramidal artifact, there were some significant differences when only GNSS data is used. For the GNSS, the standard deviation is 0.35 mm. For the RTK, the standard deviation is 0.20 mm. In all cases using the scale bars to define the scale, the errors standard deviations are the same, 0.18 mm. For the artificial defects surface comparisons, presented in Figure 10, the use of RTK or scale bars have similar results, and the errors standard deviation is remarkably similar, between 0.13 mm and 0.15 mm, and a higher result for GNNS (σ: 0.29 mm), as expected. It is important to note that scale errors and random errors are presented simultaneously in these analyses, but for the GNSS case the scale errors are evident by the red and blue areas that should be more randomly distributed and predominantly green in the absence of scale errors. In any case, it was possible to identify the artificial defect in the pipe with good quality and with much more detail if compared with a presential inspection using industrial climbing.
An evaluation was also carried out between the measured RTK/Scale surface and a best fit cylinder, thus allowing the artificial defects of the PVC pipe to be visualized, as seen in Figure 11. In real inspections, this kind of analysis should be made to detect and quantify the defects that may be present in the riser being inspected. Figure 11. The evaluation of the measured pipe by surface comparison between RTK and a fitted cylinder. Table 5 shows the root mean square error for each coordinate for all the cameras and the total error including root mean square error for X, Y, Z coordinates for all the cameras, respectively in Longitude, Latitude, and Height directions. These errors represent the RMSE between camera station given by the GNSS or RTK data and the optimized camera stations after bundle adjustment. The magnitude of the errors are as expected, including greater errors in the Z direction, but are smaller than those specified in the RPA datasheet (Yuan et al., 2009 To further analyze the error's behavior for the GNSS and RTK data, position information was investigated for the RPA in a static position on the ground. The GNSS presented the position error with a spatial standard deviation of 1428.4 mm while the GNSS+RTK presented a spatial standard deviation of 8.9 mm, as expected. In flight, the positioning uncertainty can increase. Figure 12 shows the 3D points in space recorded over time with static RPAS. For the RTK positions (right image), it is possible to observe a more random distribution of the position errors; and for the GNSS data (left image), it is possible to see a position drift over time. This behavior of the GNSS data can also contribute for the scale errors presented in the measurement results found for the GNSS strategies.

CONCLUSION
This paper investigated the influence of RPAS (also known as drones) positioning error on close range photogrammetry for industrial inspection on non-prepared scenes. The measured scene seeks to imitate the conditions found in an offshore riser inspection. Taking advantage of the position and orientation information provided by the RPAS, these data were used to reconstruct and scale the scene. The analysis was based on four different ways to scale the reconstructed scene: using only GNSS data, GNSS and scale bars, RTK, and, finally, using RTK and scale bars. These different strategies were used to process the same set of 50 acquired images in a planned network of camera stations with self-calibration. The measured scene includes a precalibrated pipe with artificial defects, a calibrated pyramidal artifact with 4 spheres, and scale bars.
The network used for image acquisition is a fusion of convergent and planar trajectories, called serpentine trajectory, and it considers the real conditions found in an offshore riser measurement task, such as limited access and safe distances, limitations with the RPA, and camera gimbal used. Using this trajectory, a good convergence is achieved around the riser longitudinal axis in the horizontal direction and in a straight line in vertical direction along the riser longitudinal axis. For five vertical lines with a Δα=15° step around the riser axis, an approximately 1:1 base to distance ratio with good convergence is achieved. No camera roll angles are used in the image acquisition as the camera gimbal has a limited roll angle (±20°), thus making the conditions not ideal for camera self-calibration. In any case, pre-calibration is not an option as the camera gimbal and lens must be changed frequently, as well as the working distance depending on the environmental conditions at the time of inspection. As expected, when only GNSS is available to determine the RPAS positioning and to geotag the acquired images, the scale error present in the reconstruction was about 5.9%. When RTK is available, scale errors were significantly reduced to 1.2% of the measured lengths in the pyramidal artifact. When scale bars were used, the scale error present in the reconstruction was 0.26%. Therefore, based on this experiment, with the settings adopted for close range photogrammetry, the results indicate that the RTK correction allows a considerable reduction in the scale error, and the use of scale bars in the scene presents a measurement with less scale error. Comparisons between calibrated and measured surfaces show that errors are significantly reduced when RTK or scale bars were used. In any case it was possible to identify the artificial defect in the pipe with good quality and with much more detail if compared with a presential inspection using industrial climbing. Further repetitions of the experiments, including variations of the environmental conditions, will be done in the future to further evaluate the navigation information influence on object space scale definition and surface comparisons.
In the future, we intend to carry out a study of additional solutions for the application of scale in the scene, such as the use of a precalibrated stereo system, which can make it possible to perform measurements without using navigation data or scale bars.