PHOTOGRAMMETRIC TECHNIQUES FOR THE RECONSTRUCTION OF UNDERWATER 3 D MODELS OF SEABED AND ARTIFACTS

The digital representation of hydraulic engineering works such as dykes or barriers is a sector that is becoming increasingly important. In general, however, the 3D relief of semi-submerged structures is still a problem. The Geomatics Laboratory in collaboration with CNR-IPCF of Messina as part of a broader project to analyze the variations of the coast lines and the seabed of the province of Reggio Calabria (southern Italy), has developed an experimental system of acquisition and processing of digital images acquired via UAV and (Remotely operated vehicle) ROV. The present note describes the design and construction characteristics of the ROV realized and used, the advantages and limitations of the use of a small and low-cost ROV, as well as the results obtained during the experimentation phase. The process of surveying of the sub-emerged structure inquired and the relative construction of a unique 3D model are also described. The two surveys were first elaborated separately and then united in a single reference system by means of the visible markers in the area straddling the sea surface. Finally, the bathymetry values obtained from the classic echo sounder system and those obtained from the model obtained were compared. * Corresponding author


INTRODUCTION
The digital representation of hydraulic engineering works such as dykes or barriers is a sector that is becoming increasingly important.In general, however, the 3D relief of semi-submerged structures is still a problem.The precision 3D survey with terrestrial photogrammetry techniques is known to be a custom in many industrial applications and for architectural, archaeological and underwater documentation.In the field of terrestrial photogrammetry, the scientific and practical problems are rather well known and supported by a substantial scientific literature of reference, differently in underwater photogrammetry, where the variables involved are different and not fully explored.
The Geomatics Laboratory in collaboration with CNR-IPCF of Messina as part of a broader project to analyse the variations of the coast lines and the seabed of the province of Reggio Calabria (southern Italy), has developed an experimental system of acquisition and processing of digital images acquired via UAV and (Remotely operated vehicle) ROV.
Specifically, with regard to the survey of the submerged part, we use an experimental, highly performing submarine drone.The system allows in fact to acquire and display in real time the backdrop and / or the object of investigation.The same is made up of two watertight chambers containing the control electronics, the power supply battery and a full HD camera necessary both for guiding the device by the operator placed on the surface and for shooting and capturing frames of the seabed.The fundamental task of the ROV is to allow an operator on the surface to interact with the underwater environment, acquiring the data from the on-board sensors and filtering them, to obtain reliable information on the environment surrounding the vehicle, as well as to integrate the sensor data to estimate the position of the vehicle with respect to the surrounding environment.In this regard ROVs can be applied in many areas, even very different from each other: monitoring the seabed for the protection of the marine environment, detection of wrecks to the detection of assets underwater archaeological sites, etc.
The present note describes the design and construction characteristics of the ROV realized and used, the advantages and limitations of the use of a small and low-cost ROV, as well as the results obtained during the experimentation phase.The process of surveying of the sub-emerged structure inquired and the relative construction of a unique 3D model are also described.The two surveys were first elaborated separately and then united in a single reference system by means of the visible markers in the area straddling the sea surface.
Finally, the bathymetry values obtained from the classic echo sounder system and those obtained from the model obtained were compared.

Emerged area
The phenomenon of coastal erosion is an increasingly reality.
To tackle the problem of coastal erosion, since the '30s Italy Institutions has implemented construction's projects aimed to defence and to contain this phenomenon.These solutions have favored the implementation of increasingly sophisticated techniques for the design and execution of coastal protection works, which have not always, however, led to the resolution of the problem, especially in the medium-long term.In fact, in many cases the rigid coastal defense works have had negative effects on the coastal dynamics, accentuating the erosion of the areas above the construction works.Specifically, the analysis has been conducted on the shore of Bocale (RC), located on the Ionian coast of the Province of Reggio Calabria and characterized by intense erosion processes, especially in the last fifteen years.
Recently, in the area of study a barrier was built (Fig. 1).The monitoring of a coastline and the section of the sea in front of it, assume a fundamental role in the study of coastal areas and consequently proves to be a valid tool for the definition of the intervention policies in coastal management and safeguard.In addition, the mapping of the seabed is of fundamental importance for the understanding of sedimentological processes.

Drone Survey
The drone model used is the Dji Phantom 4 pro, equipped with a camera able to shoot 20 Megapixel photos and with 300 Watt engines (Fig. 2).In order to obtain a correct UAV survey, the metric approach that needs to be followed concerns the data acquisition: the images, the coordinates, and the altitude of the camera centre.
To obtain a high quality ordered dataset and produce a model with high defined characteristics, the image acquisition plan was divided into three steps: definition of image acquisition plan type, definition of Ground Sampling Distance (GSD), and definition of image overlap.The Image acquisition type was set to automatic waypoint flight mission to create the orthophoto, so that the UAV perform an automatic flight.
The GSD calculation define the height of flight according to the required definition (cm/pixel) of the model.To obtain centimeter precision (GSD < 1) at comfortable flight height, considering Dji Phantom 4 camera specs (Sw = real sensor width = 12,8333 mm, FR = real focal length = 8.6 mm, imW = image width pixels 5472, imH= image height pixels 3648) the height flight was set to 30 meters for vertical flight.The overlap between two consecutive images, that determined the image acquisition rate at fixed UAV speed, was set to 80%.The image rate acquisition calculation for front overlap, is made automatically by Immagina 2000®.
Moreover, the software calculates the image acquisition plan and mission settings automatically, defined the following parameters: flight eight (and consequentially GSD), overlap (%), and area to be mapped.

Figure 2. Drone Survey landing
To better circumscribe the survey area, it was decided to plan two flight schedules with different heights (25 m and 30 m respectively), images acquisition overlap set at 80%, camera incline at 90°.Since the area had easily identifiable elements (whose coordinates were acquired through GPS), we proceeded to use them as markers.
We took a number of shots consistent with the generation of the model, integrating, where necessary, the shots taken by the UAV with additional shots from the ground (Callow et al., 2018).

3D elaboration
The acquired photograms were processed using the Agisoft Photoscan software.The workflow is completely automatic both as regards the orientation of the images and for the generation and reconstruction of the model.This condition led to an optimization of processing times ensuring good performance of the machine/software complex (Balletti et al., 2015) (Canciani et al., 2003).
The phases of the elaboration were the following: 1. Align photos (photo alignment), This is the most important phase of the entire photogrammetric process.In this phase, the software aligns our photographs with each other, calculating their position in space and reconstructing the so-called gripping geometry.Then, through a process of geometric triangulation, calculate the position in space of the elements present in the photographs.It follows that the quality of this alignment derives the goodness of the final 3D model.The result of this phase is a sparse point cloud.Following the alignment, measurement of the control points takes place, to register the block in real world coordinates, in the reference system defined by the control points.
2. Build Dense Cloud, Through this phase a dense cloud is constructed using dense image matching algorithms.These are subdivided into algorithms that use a stereo pair to find matches and those that identify them in multiple images; 3. Extraction mesh, which consists in generating a polygonal model based on the newly created dense cloud.The mesh is a subdivision of a solid into smaller solids of a polyhedral shape; 4. Build texture, instead allows to obtain the 3D representation of the work under investigation.The last step is the scaling of the model, that is the assignment of the correct metric dimension to the model, in order to make precision measurements on it (Fig. 3).

ROV survey in submerged zone
The ROV used to carry out the survey consists of two watertight chambers, the first is placed at the bow of the ROV containing both the control electronics and a full HD camera that can be used both to drive the device by the operator(placed on the surface) and to shoot and take photographs of the backdrop; the second chamber instead is placed aft and contains the battery supply.

Figure 5. Experimental ROV used in the survey
The control electronics consists of a Raspberry PI 3 that deals with the communication with the surface, the video streaming and the automatic calibration operations.The ROV is characterized by various on-board sensors, which are a nineaxis IMU (three accelerometers, three gyroscopes, three magnetometers), a pressure sensor (located outside the sealed chamber) and an integrated odometer in the tail thruster (Fig. 4, 5) (Barrile et al., 2018).
Thanks to the use of this experimental system, a photogrammetric survey of the backdrop was performed, displaying it in real time on a PC (placed on the boat) connected to the ROV by means of a suitable cable (Fig. 6).The ROV survey has been carried out starting from a depth of 4 m right next to the emerged barrier.The image acquisition takes place under low natural lighting conditions.Additional light sources were used (ROV's light).Due to a number of difficulties during the acquisition process leading to low quality images (mainly motion blur), a selection process is applied in order to select the best images to ensure a most accurate result.A grid of known dimensions assisted the ROV in order to correct the effects of distortion obtained in water and also was used as GCPs in order to orient and scale the model in a reference system.(Skarlatos et al., 2012).

Image enhancement
The underwater colours light depends on factors as the deepth, the season, the water surface conditions, the time of the day, the cloudiness of the sky, the presence of marine vegetation or the soil type.
Moreover, the effect of refraction (depending on a number of parameters such as depth, temperature and salinity) can cause unstable modelling effects, so the intrinsic parameters of a submerged camera change significantly in comparison to the primary values ( Henderson et al., 2013) (Lavest, et al., 2003) The sunlight that penetrates from the air-water interface into the water is altered from several causes and more depth increases, more the lower temperature colours disappear.
The reflections coming from the surface create light games that alter the colours.The framed scene is generally flattened, not balanced and slightly contrasted.These effects are accentuated above all as the distance between the sensor and the object increases.So all image-based applications like 3D need an image enhancement with particular regard to colours correction.Many imaging techniques have been proposed in the literature to produce a more realistic image, based on image restoration techniques or qualitative.
In this work for colour correction of underwater we used the algorithm proposed by Bianco et al. 2015, where colour correction of underwater images is performed by using lab colour space.In particular, the chromatic components are changed moving their distributions around the white point (white balancing) and histogram cut-off, and stretching of the luminance component is performed to improve image contrast.Main constrains of this method are the grey-world assumption and the uniform illumination of the scene (Bianco et al., 2015).Moreover, images were processed with Adobe Photoshop in order to enhance the contrast and sharpness and find the best solution for each specific application.Automated algorithm "Find Dark and Light Colours" was used for this correction.According to the software, this algorithm analyses the image in order to find dark and light colours and uses them as the shadow and highlight colours.The option "Snap Neutral Midtones" was also checked.This adjusts the midtones so that colours that are close to neutral are mapped to the target neutral colour.(Drap et al., 2007)

3D Model Elaboration
The process to create the immersed 3D model is equal, net of image processing, to that illustrated in paragraph 3.1 (Fig. 8).The positioning of a "rudimental and low cost" grid on the sea bottom (where both the distance from the barrier and the depth are known) allows to scale the model, making it consequently available in metric terms (Fig. 7).

Figure 7. Rudimental grid
Regarding the model positioning, we proceeded connecting a buoy (on the water surface) to each point of the grid placed on the sea bottom and then to measure buoy's planimetric coordinates through Leica GS08 plus GPS.
Clearly to obtain the coordinates of the grid points we changed the altitude value with the depth value (prior measured trough the length of the rope needed to maintain the buoy on the surface).Then we link the 3D model using the grid coordinates thus obtained and the topographic survey on the shore used to georefer the barrier.(Liarokapis et al., 2017 ) Figure 8. 3D model of seabed

Models integration
As previously illustrated, the two measurements have been elaborated separately and georeferred individually with automatic orientation procedures.We then proceeded to merge the two models (emerged and submerged) into a single reference system using points clearly visible and identifiable in both the models located both in the area straddling the sea surface, and between the different parts that make up the submerged final model (the dunk chunk) (Remondino et all 2012.).Subsequently automatic image matching techniques were employed for generating dense clouds of points both of the immersed and emerged part, obtained in the final reference system.The clouds were then triangulated obtaining a global 3D mesh (Fig. 9).
Figure 9. 3D model Evaluated the reliability of the obtained 3D model, we thus proceeded to the "local" determination of the bathymetry of the area subject of investigation (Skarlatos et al., 2012).

Echo Sound Survey and result comparison
The comparison was made by calculating the differences in altitude between the survey by the ROV and the points acquired through survey by drone and the points acquired through traditional multibeam survey.In order to evaluate the quality of the obtained results, a bathymetry has been created through the use of a Multibeam echo-sounder that, as known, is a technology that allows to have a high definition mapping of the sea bottom thanks to the amount of data which is possible to acquire.
The physical phenomenon on which it focuses consists on acoustic sonar that measures the return echo coming from the sound emitted by the transducer and reflected by the sea bottom.From this it gathers that the resulting survey consists on a stripe whose axis continues to follow the boat route trace and which develops perpendicularly to this axis for amplitude that depends on the sea bottom depth and the angular gap on which the beams are emitted.The frequency with which these are emitted depends on the conditions in which the survey is carried out.In each instant the beam emitted and detected by the transducer of a muti-beam must be correctly arranged in relation to the system used to carry the surveys out.For this reason must be referring to an integrated system consisting of a RTK GPS system (which results essential to georeferencing each single distance measured by the beams of which the transducer is composed), an acquisition and elaboration data system and auxiliary instrumentations (whose task is to correct the data detected by the echo-sounder through the information concerning the orientation of the multi-beam, the transducer movements caused by the wave motion, rolling and pitching of the boat).The data measured by the single instruments are managed by navigation and acquisition system which provides to associate to each distance measured by the transducer a correct X, Y and Z values in the used reference system.The multi-beam data acquisition criteria is based on the total coverage of the investigation area, coverage that is obtained through the definition of a squared mesh grid having a certain number of spatial data as base detected through GPS (Barrile et al., 2018).Every time that the system detects a point which is includes within this cell, it gives back the average of all measurements and shows the average depth for that specific surface unit.The result is shown in Fig. 10, 11 and 12.Although with all the limitations of the case (being the methodology experimentation adopted only in the initial phase) the comparison of the measurements obtained by the ROV 3D model and the multi-beam system highlights (Fig. 7) average deviations of the order of 50 cm.
The detailed 3D modeling thanks to the using of the experimental ROV has allowed to have a good valuation of the submerged shore and it will contribute for sure, to integrate the hydraulics maritime studies.

Conclusion
The particular attention that is given to the erosive phenomenon (that subtracts sand and sediments causing a lowering or flattening of the seabed) is justified by the strong construction along the Italian coasts.At the same time, constructions designed to block the phenomenon require careful planning and control.
The surveying process of the inquired sub-emerged structure gives to the planner and designer a perfect representation of the state of the art.It is possible thank to the construction of a unique 3D model (seabed and barrier).
The technique proposed for the calculation of bathymetry even if it has a low metric precision with an average error of 31 cm is to be considered for the type of relief and the final use, comparable to a traditional survey.

Figure
Figure 1.Study Area

Figure 6 .
Figure 6.Display of ROV acquisition