ACCURATE SCALING AND LEVELLING IN UNDERWATER PHOTOGRAMMETRY WITH A PRESSURE SENSOR

: Photogrammetry needs known geometric elements to provide metric traceable measurements. These known elements can be a distance between two three-dimensional object points or two camera stations, or a combination of known coordinates and/or angles to solve the seven degrees of freedom that lead to rank deficiency of the normal-equation matrix. In this paper we present a novel approach for scaling and levelling to the local vertical direction an underwater photogrammetric survey. The developed methodology is based on a portable low-cost device designed and realized by the authors that uses depth measurements from a high resolution pressure sensor. The prototype consists of a data logger featuring a pressure sensor synchronized with a digital camera in its underwater pressure housing. The modular design, with optical communication and synchronization, provides great flexibility not requiring the camera housing to undergo any hardware modifications. The proposed methodology allows for a full 3D levelling transformation comprising two angles, a vertical translation and a scale factor and can work for surveying scenes extending horizontally, vertically or both. The paper presents the theoretical principles, an overview of the developed system together with preliminary calibration results. Tests in a lake and at sea are reported. An accuracy better than 1:5000 on the length measurement was achieved in calm water conditions.


INTRODUCTION
Different kinds of 3D measurements are collected in underwater photogrammetry. Whether it be a simple distance between two 3D points, for example to measure the length of a coral branch, or more elaborated ones, such as multi-temporal digital elevation models for the documentation of an archaeological excavation or 3D reverse modelling for structural analysis in subsea metrology, photogrammetry needs known geometric information to provide metric traceable measurements that can be analyzed and interpreted by researchers, scientists, engineers and technicians all over the globe. As explained in photogrammetry textbooks (Kraus, 2011;Mikhail, 2001;Luhmann et al., 2013), these known elements might be a distance between two 3D points or two camera stations, or a combination of known coordinates and/or angles to solve the seven degrees of freedom for absolute (or local) orientation. Although these basic concepts are implemented as common photogrammetry tasks above the water, where ubiquitous solutions exist (e.g. GNSS+INS direct georeferencing, collection of ground control points -GCPs via robotic total stations or laser trackers, invar scale bars certified up to few microns of uncertainty), when it comes to underwater photogrammetry, the available solutions might be too expensive, not enough accurate or sometimes not feasible at all for safety constraints.

Underwater scaling via GCPs and direct measurements
In underwater archaeological documentation, the measurements of GCPs have been traditionally carried out using direct methods based on trilateration combined with depth values from dive computers and azimuth directions from a compass. These * Corresponding author surveying operations, carried out by divers, are very time consuming and, according to non-decompression limits, only safe in relatively shallow water. Moreover, an almost horizontal planar configuration of measured distances, very typical in archaeological excavations (e.g. shipwrecks), provides very uncertain Z coordinates (Skarlatos et al., 2019). More accurate direct measurements by divers have been recently demonstrated (Nocerino et al., 2020) combining trilateration and geometric levelling underwater with lasers for collecting sub-centimeter accuracy GCPs for 3D photogrammetric measurements aimed at multitemporal analysis for ecological research. Yet, the even greater complexity of these underwater surveys carried out by divers restricts its use only to areas of limited extension and at shallow depths (<20m). At depths greater than 60m, direct measurements even by technical divers with rebreathers become very risky and ineffective and is only carried out by professional saturation divers . For this reason, simpler methods based on scale bars and spirit bubble levels are preferred. These are set on stable fixed positions at the seabed and used to scale and set up the local datum. Nevertheless, the use of scale bars for photogrammetry purposes does not provide an independent control over possible deformations of the photogrammetric model that, as result, can still be warped showing 3D polynomial deformations. Direct measurements by underwater robots, such as ROVs, require great maneuverability and special adapters to overcome the limited dexterity of a typical ROV manipulator; consequently, acoustic based trilateration measurements are preferred. However, their elevated costs are only sustainable for industrial measurements, such as in the Oil&Gas industry (Guilloux, 2014). The above-mentioned solutions require, to some extents, an interaction through contact with the object to set up and fix the scale bars, signalize the GCP points, and measure them, which in many applications might not be feasible. On the other hand, noncontact underwater photogrammetry solutions have been known for long time using for example underwater multi camera systems or laser pointers.

Underwater scaling via stereo-cameras and laser pointers
Synchronized stereo and multi camera systems provide great flexibility and the ability of measuring even dynamic scenes, such as moving fish for aquaculture applications. However, these benefits come at the increased cost of multiple sensors, more power, and a heavier and more cumbersome capturing system. Moreover, the scaling accuracy depends on the baseline length, calibration method of the multicamera rig and its mechanical stability. Laser pointers have been a popular low-cost solution for providing a scale (Istenič et al., 2020): they require the relative orientation of the laser direction with respect to the camera and custom algorithms to detect the laser spot in the image to perform forward triangulation. The scaling precision bounded by laser to camera baseline, mechanically stability of the rig, presence of backscattering and speckle artefacts, especially in turbid water, are among the most limiting drawbacks of such a solution.

Our approach: underwater scaling via pressure sensor
An approach based on pressure sensors for online approximate scaling of a ROV trajectory from visual odometry was proposed by Crueze (2017). Here we present a similar technique based on a new portable low-cost solution that uses a single camera in its underwater pressure housing, and a synchronized data logger featuring a pressure sensor. Pressure data are converted into depth measurements. Differently from the previous study (Crueze, 2017), we propose a calibration method that considers sensors offsets and water density and can work also without an attitude and heading reference system (AHRS). Moreover, the proposed pipeline allows for a full 3D levelling transformation with scale factor and can work both for surveying scenes extending horizontally, vertically or both. In the developed prototype, pressure data are synchronized with the image acquisition through optical signals, generated, sent, and received through water by inexpensive microcontrollers (e.g., Arduino 1 ). The modular design and 1 https://www.arduino.cc/ optical communication provide great flexibility not requiring the pressure housing to undergo any hardware modifications. Therefore, the prototype system works with any commercial camera that features a hot shoe contact for firing a strobe in its underwater pressure housing. The prototype ( Figure 1a) was mainly thought for, and has been tested by divers ( Figure 1b) although its installation on a small ROV is possible.
The main concept behind the use of depth measurements for scaling and setting a local datum with Z coordinate coincident with the local vertical direction is quite straightforward, being similar to direct georeferencing using a GNSS receiver only (without INS). The main difference is that being the Z coordinate the only observations provided by the depth sensor, only the scaling and "levelling" with respect to the local vertical direction is possible. The method has its theoretical foundations in analytical photogrammetry, where image orientation and triangulation were solved using control points coordinates partially known in planimetry or altimetry. In our approach, the free network photogrammetric solution is first rotated in an approximately levelled orientation. Then, the unified approach of least squares Helmert transformation (Mikhail, 1976) follows to transform the photogrammetric exterior orientation into the final levelled coordinate system. The transformation minimizes the squares of Z differences between the photogrammetrically estimated coordinates and measurements from the depth sensor. The method is able to transform to a local datum and scale the free network solution with some requirements on the imaging geometry (for example at least four non-coplanar images). To take into account the displacement of the pressure sensor center with respect to the entrance pupil of the camera, a lever arm calibration is required. The proposed method is fully contactless and can be applied to most typical underwater photogrammetry surveys such as in archaeological documentation, ecological research or subsea inspection and metrology. Also, compared to other low-cost methods based on laser scalers, it provides the local vertical direction.

4 DOF transformation with a pressure sensor
The measurement of underwater depths using a pressure sensor is based on the principle of hydrostatic pressure. The hydrostatic pressure is the pressure exerted by a fluid at equilibrium at a given point within the fluid due to the force of gravity. Assuming the pressure sensor to be placed underwater at a depth Dsensor , the measured pressure will be the sum of the atmospheric pressure P0 at the water surface and the pressure exerted by the water P1, which is a function of the water density ρ and gravitational acceleration g. The depth Dsensor can be computed as follows: Where g depends on the latitude and altitude and ρ depends on water salinity. Typical standard values are: g ≈ 9.80665 m/s 2 ; ρ ≈ 1029 kg/m 3 for salt water and 1000 kg/m 3 for fresh water.
In reality, the camera center is at heigh difference Δz measured along the vertical direction with respect to the pressure sensor center, as shown in Figure 2a. The value Δz is a function of the exterior orientation of the camera and of the lever arm offsets [Δxcam, Δycam, Δzcam] between the camera entrance pupil and the pressure sensor center (Figure 2b). For accurate photogrammetric measurements, these offsets must be known either through a laboratory calibration process, where the offsets are measured using an optical collimator, or computed analytically through a self-calibration using least squares adjustment. The scaling and levelling of the results of a photogrammetric bundle adjustment using pressure measurements is based on a simple geometric principle, illustrated in Figure 3. The water, assumed to be calm (no waves) and static (no currents), is a horizontal surface. This is by definition orthogonal to the vertical direction along which depth D is measured. If the vertical direction is unknown (arbitrary datum), the water surface may be tangent at any point to the sphere of radius D (Figure 3a) and its orientation is therefore unknown. Figure 3b,c,d show a 2D representation of three camera centers O in a triangular configuration with associated depth measurements, represented as circles in 2D. The water surface, as said before, must be tangent to all the three circles. Because the initial results of the bundle adjustment are neither scaled nor levelled, there does not exist a line tangent to the three circles ( Figure 3b). The distances between the circle centers are then isotropically scaled until the three circles become tangent to the same water line (scale factor of 1/2 is found in the example) (Figure 3c). A rotation is then applied to make the water line horizontal (Figure 3d). In 2D space, at least three depths measured from three noncollinear points are necessary, in 3D space four depths from four non-coplanar points are necessary to level and scale the 3D photogrammetric model. Analytically, this corresponds to determine four degrees of freedom (λ, ω, φ, Z0) of a 3D Helmert transformation from the arbitrary scaled local reference coordinate system of the bundle adjustment (BA) to the scaled and metric vertical system (V). The remaining three degrees of freedom (X0, Y0, κ) of the Helmert transformation are arbitrarily set equal to zero: Figure 3. Geometric principles of scaling and levelling with depth measurements. a) A depth measurement identifies a circle of position, centered in the camera and with radius equal to the measured depth value. The water level can be tangent at any point of the circle. b) Three different depth measurements define three circles of position. c) For the water line being tangent to the three circles, the triangle defined by the three tangential points is isotropically scaled. d) A rotation is applied to make the water line horizontal.
where are elements of the rotation matrix• , function of the two unknown Euler angles ω,φ. λ is the unknown scale factor, and (XD, YD, ZD)BA are the threedimensional coordinates of the pressure sensor in the bundle adjustment reference system. The pressure sensor coordinates are function of the camera exterior orientation parameter and lever arm offsets according to the following equation: where is the rotation matrix that transforms 3D coordinates from the camera to the bundle adjustment reference system and is a function of the exterior orientation of the camera. A minimum of four non-coplanar images (and associated depths) is necessary to estimate the 4 unknowns λ, ω, φ, Z0 from equation (3). As usual for any surveying disciplines, redundancy is mandatory and more than the minimum number of observations is collected giving an overdetermined system that is solved, after linearization, using the least squares method.

Prototyped system with underwater pressure sensor
The developed prototype (Figure 1) consists of two parts, as sketched in Figure 4: 1. camera trigger; 2. pressure sensor data logger.
The camera trigger module is made of a microcontroller plus electronics mounted inside the camera housing on the hot shoe of the camera, in place of the strobe. Its task is to read the signal from the camera hot shoe whenever a photo is shot and then to send an optical signal through the optical fiber port to the underwater data logger. A real time clock plus a micro SD card permit data logging on the camera housing side (e.g. shutter time duration, synchronization events). This design allows for compatibility with most underwater camera systems without any modification to the camera housing. The pressure sensor data logger features a microcontroller, a pressure sensor, a thermometer, a real time clock, a micro SD card, an OLED display for communication and some user input buttons. The electronics is enclosed in a waterproof housing that can resist to a maximum depth of 60 meters. The depth sensor (in blue in Figure 1a, 4) is in contact with water and connected to electronics in the housing through a bulkhead connector. It has a resolution of 0.2mbar and absolute accuracy of 20 mbar. The synchronization of the two modules via optical signals is within a fraction of millisecond, measured in laboratory using an oscilloscope.

Software
Before each survey, an initialization is performed to measure the pressure P0 and synchronize the real time clocks of the two modules via optical communication. After each photogrammetric survey, a software reads the data in the two SD cards (one from the camera and one from the pressure sensor) and associates the pressure data to each photograph writing the relevant information as comment within the EXIF of the image file. Another software is used to compute the scale and levelling 4 DOF transformation parameters according to the least squares principle as described in Section 2.1. The software takes as input the camera positions obtained from the bundle adjustment as performed in photogrammetric or SfM applications and outputs the transformation parameters with least squares summary statistics (e.g RMS of residuals, standard deviation of parameters).

EXPERIMENTS AND VALIDATION
The developed prototype was tested both at sea and in lakes, thus with different water density respectively used for salt and fresh water. The general functioning was verified up to a depth of about 42 meters. In this section we report the results of two camera calibrations carried out using an Olympus OM-D E-M5 mark II mirrorless camera featuring a 16MP Four Thirds CMOS sensor and a M.ZUIKO 9-18mm zoom lens set at 9 mm. The waterproof housing was the PT-EP13 mounting an AOI optical glass dome port. The aim of the experiment was to: 1. capture a camera network characterized by strong geometry, redundancy and angle and depth diversity to obtain accurate and reliable exterior orientation parameters; 2. apply the developed procedure for scaling and levelling; 3. analyze and discuss the least squares adjustment results in calm and more rough water conditions; 4. assess the scaling accuracy by comparing the measured length of scalebars of known length (measured in laboratory with an uncertainty better than 0.1 mm) deployed on the seafloor.

Sensor offsets calibration
The calibration of sensor offsets [Δxcam, Δycam, Δzcam] was performed in laboratory on an optical breadboard using the procedure presented in Menna et al (2018). The estimated accuracy was better than 3 mm.

Water density and gravitational acceleration
Equation (1) requires calibrated values of water density ρ and gravitational acceleration g. Very good approximations can be obtained from formulas (e.g. international gravity formula), tables or local measurements (water density). An alternative and easier way consists in performing an in situ calibration following Garda Lake, Italy La Ciotat, France Figure 5. Image datasets captured respectively in the lake (upper row) and at sea (lower row) to test the developed scaling and levelling procedure based on hydrostatic pressure measurements. The camera network is shown in the right column.
the procedure described in Section 2.1 with a scaled photogrammetric survey. In this case, a scale factor is found to correct the estimate of the product ρ⋅ g with respect to the nominal value and can be reutilized in similar water conditions (for example water type and temperature).

Image calibration datasets
The two camera calibration datasets ( Figure 5) were acquired respectively in the Garda Lake, Italy and at sea in the bay of la Ciotat, France between November and December 2020. The image network consisted of about 90 images including convergent and rolled photographs taken at different depths. The value of the product ρ⋅ g at the denominator of equation (1) was obtained from previous calibrations done in similar conditions (water type and temperature) using a scale bar as described in Section 3.2. In both cases two one meter-long scalebars were placed orthogonally to each other on the floor.

Results
The image datasets were oriented in Agisoft Metashape using self-calibration for standard Brown/Beyer parameters (Gruen & Beyer, 2001). The exterior orientation parameters were exported to compute the 4-DOF parameters transformation as described in Section 2.1. The two scalebars were used to assess the accuracy in measuring the length distances. The computed length distance lM from the bundle adjustment, scaled using the pressure sensor data, were compared to the calibrated reference length distance lR from laboratory giving as difference the length measurement error LME and relative length measurement accuracy RLMA: Each 1 m long scale bar features 6 circular target (3 for each end), enabling a total of 18 length distances (ca. 1 m) to be measured.  Table 1. Image datasets at sea and in lake and results of the 4 DOF scaling and levelling transformation. Table 1 summarizes the main datasets characteristics, the precision results of the four computed parameters λ, ω, φ, Z0 using least squares adjustments and the achieved accuracy metrics in terms of length measurement error LME and relative length measurement accuracy RLMA.

DISCUSSION
The results in Table 1 show a high potential accuracy achievable with the proposed method based on pressure measurements. As expected, measurements at sea provide less precise results given by the fact that the reference level measurement for the depth (water level) is continuously changing. Nevertheless, although the precision for the parameter λ was about 1.5%, we obtained a RLMA of about 0.1% of the measured length, indicating a good accuracy. In non-ideal water conditions (wavy sea), it must be expected that the accuracy of the proposed scaled method may exceed its precision. Indeed, under the assumption of a regular sea, waves can be modelled as a zero-mean sinusoidal function. When acquiring a large number of images, the average level for pressure measurements tends to equal the zero-mean water level, providing high accuracy. On the contrary, the spread of depth values can still be high, causing low precision.

CONCLUSIONS AND FUTURE WORK
The paper provided an overview of the developed system and its theoretical background for a full 3D transformation comprising two levelling angles, a vertical translation and a scale factor. In its simplicity the method results very effective as it is fully contactless and can work for surveying scenes extending horizontally, vertically or both. The system calibration and characterization were presented and validated through experiments conducted in different underwater scenarios. Two different data acquisitions were performed in calm fresh water (Garda lake, Italy) at depth of about 14 meters and at sea (near Marseilles, France) at a shallower depth of about 4 meters with rougher sea surface conditions. The empirical scaling accuracy better than respectively 1:8000 in calm water in a lake, and 1:1400 at sea, were verified using calibrated scale bars. In calm water such as in a lake, under the ice, in caverns and caves, potential accuracy better than 1/10000 on the scaling and 1/100 of degree for the local vertical direction can be expected with a pre-calibration of the product of the two parameters ρ and g. In the future we will extend the computation to the offsets [Δxcam, Δycam, Δzcam] as unknowns in the least squares procedure. An experimental procedure will be also put in place to assess the angular accuracy of the determined levelling parameters. We will analyze more in depth the effect of errors caused by waves and will consider the addition of an inertial unit for redundancy and to further improve robustness and accuracy of the estimated parameters.