3 D MODELING OF RIO MIERA WRECK SHIP MERGING OPTICAL AND MULTIBEAM HIGH RESOLUTION POINTS CLOUD

3D reconstruction and virtual reality (VR) technology provide many opportunities for the documentation and dissemination of underwater cultural heritage. Advances in the development of underwater exploration technology have allowed for the first time to accurately reconstruct a complete 3D model of the cargo Río Miera in the Cantabrian Sea. Sunk on December 6, 1951 after a strong collision, the cargo ship Río Miera rests on a sandy bottom about 40 meters deep, very close to the Cantabrian coast. Located in an area of strong currents is a classic objective of the region for the most experienced divers. The survey was carried out this summer in R/V Ramón Margalef of the IEO, acquiring acoustic data with multibeam echo sounders and hundreds of images acquired by a remotely piloted underwater vehicle. The campaign is part of the PhotoMARE project Underwater Photogrammetry for MArine Renewable Energy. This work describes the workflow regarding the survey, images and acoustic data acquisition, data processing, optic 3D point cloud color enhancement and acoustic and optic dataset merging procedure to obtain a complete 3D model of wreck Río Miera in Cantabrian Sea. Through this project, Spanish Institute of Oceanography – IEO have advanced – combining acoustic and image methods in the generation of 3D models of archaeological sites and submerged structures.


INTRODUCTION
Underwater photogrammetry is showing as one of the most promising techniques for mapping underwater archeological sites.The 3D reconstruction of complete models of these historical sites will allow us to advance in their knowledge, conservation and protection.Until recently, the application of this type of technology was limited to shallow areas through the diver based approach (Mahon et al., 2011;McCarthy and Benjamin, 2014).
However, the use of ROVs and other underwater vehicles have become a common practice.These remotely manned vehicles are instrumented with different sensors (cameras and acoustic sensors) and they have allowed to apply underwater photogrammetric approach even in deep-sea (Gambin et al., 2018;Drap et al., 2015;Foley et al., 2002).
Multibeam echosounders typically give good results at long distances and in presence of bad visibility.Acoustic systems can cover a large extension of sea-floor, but the resulting 3D data are affected by low resolution and color information must be restricted to represent depth information.Despite, passive optical sensors achieve better spatial and color resolution, but their successful depends on the presence of clear water and their scope is limited.
Underwater optical imaging suffers light absorption and color alteration induced by the water.Absorption of lower wavelengths of radiation and turbidity cause greener imaging and produce loss of contrast.These problems limit the use of optical techniques, and require a proper data acquisition methodology.Sometimes the imaging quality is inadequate so the use of algorithms to enhance the underwater image radiometry is necessary to restore actual color of scenes (Agrafiotis et al., 2018).However, the changes made to the image radiometry applied in individually mode (scene by scene), can affect to the photogrammetric bundle adjustment producing difficulties or even making impossible to fit imaging dataset in a single block.
The integration of dataset acquired by acoustic and optical systems to get a complete 3D model of archaeological site can improve the way to document underwater cultural heritage sites.Optic and acoustic data fusion is an approach for mapping underwater archaeological sites and other sea-floor features which has a growing interest among the scientific community.In recent years different studies have been carried out with the aims at developing innovative tools which link underwater photogrammetric and acoustic measurements (Drap et al., 2018;Hurtós et al., 2010;Moroni et al., 2013).
Improving acoustic acquisition resolution and increasing the number and quality of available images allow merge both data types and get a powerful tool for no-destructive archaeology in underwater environment.In the following case study, enhancement of colored 3D point cloud got from SfM (Structure-from-Motion) technique and the fusion with acoustic point cloud is presented to improve the 3D reconstruction of shipwreck site.This allows us to improve the knowledge of these archeological sites in order to be studied, conserved and protected.

Wreck site
The Río Miera (Fig. 1) is the most representative wreck of the coasts of Cantabria region (North of Spain).It was an English coal cargo ship, with the machine aft and three poles.It was built in 1920 in Lekkedker by Van Duijuendifk, it had 744 tons of registration.Nowadays, with 55 meters of length and 9 meters of beam the freighter gives refuge to numerous species of fish.
In 1929 it was sold to the mountain shipping company and changed its name to Río Miera.It transported mineral from Castro and Bilbao, coal from Asturias and salt from Cádiz.The study area is located at 15 nautical miles from the coast in a sedimentary zone (Fig. 2).
Figure 2: Spain map with wreckships in red points and study area in yellow box; wreck site in the Cantabrian Sea; and an photograph of Río Miera.

Survey description
The data sets analyzed in this study were acquired in wreck site during the PhotoMARE-2018 survey, carried out with the RV Ramón Margalef.The video sections were acquired using Politolana ROTV (Remotely Operated Towed Vehicle).
Politolana sled (Sánchez et al., 2017) is a robust submarine towed vehicle, designed by the Santander laboratory of the Spanish Institute of Oceanography (IEO), to study the deep-sea floor using photogrammetric methods.This vehicle acquires simultaneously still pictures and HD video, and synchronizes it with geographical coordinate and environmental variables.
Five video-transects were recorded in July 2018, with a FullHD video-camera (Sony HD-700-CX) 2 without artificial illumination and 3 with two LED light (12600 lumens / 6000 ° Kelvin).The system was equipped with 2 parallel laser beams separated by a constant distance of 20 cm.This distance provides a method of scaling and validating the final model.
Absolute positioning of vehicle is provided by Kongsberg HiPAP 502 Super (Ultra) Short Base Line (SSBL).This fully omni directional system can be pointed in any direction below the vessel, as the transducer has the shape of a sphere and an operating area of 200°.The ROTV has a transponder operating acoustically to be positioned.
Acoustic data were acquired by the multibeam echosounder Simrad EM-710 in 3D scanning operating mode.This mode implies the acquisition of data by rotating its sensor head from -10º to 10º with a step of 1º.The acquisition lines were made with Dynamic Positioning at very slow speed (0.1 knot), which increases the data density and resolution.

Photogrammetric reconstruction
Two opposite directions video-sections were decomposed in thousands of overlapping images with geo-position attached; and processing was performed using photogrammetric Pix4D Mapper Pro software (Pix4D SA, Switzerland).The sections without artificial illumination ware selected because they were the most uniform and regular of the dataset.
The triangulation algorithm is based on binary local key point, searching for matching points by analyzing all images.Those matching points, approximate values of image position and orientation provided by SSBL and Politolana telemetry system, are used in a bundle adjustment to reconstruct the exact position and orientation of the camera for each acquired image.
The focal length, principal point and radial/tangential distortions were set as initial theoretical values and the bundle adjustment processing determines the final internal orientation parameters of the camera.With this complete automated integration of tie points measurements, camera calibration, and position data of cameras, software gives as results 3D dense point clouds (Fig. 3) and 3D model mesh.

3D optic point cloud color correction
The deficiencies that underwater images usually suffer are well described.The most obvious and that may affect the subsequent exploitation of these images are the blue-green color (producing greenish-blue scenes) of the images and the obvious loss of contrast that affects the discrimination of details.Added effects that can be produced by the turbidity of the waters and the presence of mobile organisms such as the presence of shoal fish are important factors too.
Due to different factors, such as the high levels of cloudiness and turbidity found during the image acquisitions, and the depth at which the wreck is located, visibility was too low.Consequently, red fraction of RGB is almost null and causes the images have a marked greenish color (Fig. 4).In the webpage of iMARE CULTURE H2020 project, we can find a freely accessible tool designed specifically for images acquired in the underwater environment.This tool implements five algorithms aimed to enhance the quality of underwater images.It is capable to automatically process a set of images within a directory, with all algorithms or just a subset of them.Two filters are applied to the Rio Miera images sequentially, so second filter is applied to the result obtained with the application of the first one.The selected filters are LAB -Lab Color Correction (Bianco et al., 2015) and ACE -Automatic Color Enhancement (Getreuer, 2012).
LAB algorithm is a color correction of underwater images by using the lαβ color space; the chromatic components are changed moving their distributions around the white point (white balancing) and histogram cutoff and stretching of the luminance component is performed to improve image contrast (Bianco et al., 2015).
The ACE method of (Gatta et al., 2002) and further developed (Rizzi, 2004;Rizzi et al., 2003) and (Bertalmío et al., 2007) is an effective color correction and enhancement method based on a simple model of the human visual system.
The improvement produced individually in the video frames acquired was evident.But the subsequent photogrammetric process (bundle adjustment) did not achieve the same result as the green images and there adjustment of some zones were fictitious; not identifying points that were the same in different successive frames as similar, producing artifacts in the 3D point cloud.
Therefore it has been decided to implement these same filters, since the improvement in color and contrast of the images is evident, on the generated 3D point cloud.Applying the filter after the photogrammetric process and not on the individual input images allow that the changes in individual frames does not affect to bundle adjustment process.To do this, a rectangular empty synthetic image is created with the same number of points as the cloud (4403x1697) and the RGB values of the cloud are inserted in each pixel, losing the spatial information of the data (Fig. 5).The image is saved in PNG format and processed applying the same filters that were tested on the individual images (LAB + ACE).Then the inverse process is performed.That is, the new RGB values are extracted from the synthetic image pixels and assigned into the 3D point cloud.

Acoustic processing data
The high-resolution bathymetric data were collected on board R/V Ramón Margalef with the software acquisition SIS (Seafloor Information System) by Kongsberg.Afterwards, data were treated with the data processing software, Caris HIPS and SIPS 9.1.
These data were processed according with a systematic and detailed methodology.Due to the continuous changes in the environmental variables of the marine environment that interfere directly in acoustic data acquisition, it is necessary to make several corrections for the navigations sensors.In this way, the first step in the processing data flow is correct the velocity sound speed profile for its implications caused by the atmospheric conditions and marine currents.Additionally, the tide corrections and sensor corrections (navigation, pitch, roll or wave) are applied.Secondly the processing is focus on depth data and processed by unique lines, followed by 3D processing on data blocks.The sounding error data were eliminated and the necessary corrections and filters were introduced.The filters and statistical algorithms such as CUBE (Combined Uncertainty and Bathymetric Estimator) facilitated the data cleaning and the compilation for a final model (Fig. 6).Usually a manual coarse fit has to be executed to approximate the two optic and acoustic point clouds.This is common in diving case that the images have been taken by divers and they have not been associated with any positioning system.In our case, both optic and acoustic point cloud are georeferenced from the ship's GPS positioning systems and SSBL system.
For fine registration CloudCompare use ICP algorithm.ICP, Iterative Closest Point (Besl and Mckay, 1992;Chen and Medioni, 1991;Zhang, 1994) is an algorithm employed to minimize the difference between two clouds of points.ICP is often used to reconstruct 2D or 3D surfaces from different scans.Essentially, the algorithm for each point in the source point cloud ('data' in CC), match the closest point in the reference point cloud ('model' in CC).Estimating the combination of rotation and translation using a root mean square point to point distance metric minimization technique which will best align each source point to its match found in the previous step.As results of fine registration (ICP) algorithm to the data cloud and the model cloud, CloudCompare software output the resulting transformation matrix in the console.

Photogrammetric reconstruction
Digital Surface Model (DSM), orthomosaic, 3D dense point cloud and 3D textured mesh of Rio Miera wreck ship have been obtained as results of photogrammetric reconstruction.These 2D cartographic layers can be included in a GIS environment, since all the information is geo-referenced in a geographic reference system (UTM-WGS84).For its part, 3D textured mesh can be spread in 3D viewer for the web, for example in Sketchfab (https://skfb.ly/6HNAr).This type of viewer allows share and embed 3D models anywhere online.The distances between parallel lasers beams (20 cm apart) are used as validation scales.Six spacing measurements are evaluated in 3D point cloud.Laser spacing set in the ROTV is 20.0 cms and the mean and standard deviation of the six control distance in cm is 19.0 ±0.43.

3D optic point cloud color correction
The frame-by-frame application of the underwater image enhancement filters provided through the iMARE CULTURE project achieve a considerable improvement of the images.The combination of both filters (LAB + ACE) restores the actual color of scenes and increases the contrast to avoid the loss of details (Fig. 7).
However, bundle adjustment does not work properly in Pix4D with the enhanced imaging.Bundle adjustment process is very sensitive to the lack of coherence between adjacent images.So the filters, even giving very good results, cannot be applied to the frames before block adjustment processing and densification of the point cloud.The color and contrast enhanced filters -through the same algorithms selected for the images-are applied to the 3D colored point cloud.Since the LAB and ACE algorithms used do not contain spatial considerations, which is very common in other types of image enhancement algorithms, the improvement of color and contrast works correctly on the synthetic image generated.After the enhancements are applied to this synthetic image, the new RGB values are restored to the points of the 3D cloud, associating them again with their spatial component (Fig. 7).The filters LAB + ACE obtain similar results with the complete 3D point cloud than individual scenes.

3D acoustic point cloud
A cloud of validated points with a very high resolution of 0, 1m resulted by this processing work.Filters and resampling were applied in order to achive the best solution.The point cloud has considered 1412987 point data.It covers an area of 31,450 m2, not only the wreck, but also the surroundings in order to characterize the zone (Fig. 9).
A quality analysis has been carried out with the raw data acquired.All the values achieve the standards required by the International Hydrographic Organization (IHO) for hydrographic surveys.The high resolution level obtained in this point cloud is due to both the high density of points obtained by the 3D scanning mode (coordinate Z), and the precision of the X and Y coordinates obtained from global positioning system (GPS) with differentials corrections in real time.For this reason, given the final centimetric resolution, this cloud of points has been considered as master cloud to carry out the adjustment of the optical point cloud.

Merging acoustic and optic 3D point clouds
Merging acoustic and optical point cloud a very accurate and complete 3D model of cargo Rio Miera has been obtained.
Given the initial fit between the two points of clouds (both in position, and in scale); the fine adjustment is carried out directly by applying the ICP algorithm using CloudCompare.
The transformation matrix (Table 2) shows very low displacement, rotation and scale values.For the interpretation of this matrix, it must be taken into account that this matrix is a composition of a rotation (a 3x3 matrix) and a translation (a 3D vector).For convenience, it can be written as a 4x4 matrix (the rotation matrix corresponds to the upper part of the 3 first columns, the translation vector corresponds to the upper part of the 4th column, and the bottom line is always filled with 3 'zeros' and a 'one').There are minimal differences in the orientation of both point clouds.For the displacements, the difference in (x, y) is (-2.86,6.2) meters and in Z (7.26) meters.The difference in scale is again minimal (1,006).
These data are reasonable and according to the nature of the datasets.The optical point cloud has been subjected to manual editing to avoid some erroneous points (noise), but this process has not been exhaustive in order to minimize manual tasks.
Besides there are some fine and vertical elements in the structure and the size of these elements cannot be detected by acoustic system (Fig. 10), so these groups of points can cause differences between datasets.It is therefore understood that these translation values could even be improved by performing an advanced edition to the optical point cloud.

CONCLUSIONS
A 3D underwater model offers many possibilities for the documentation of underwater cultural heritage, thanks to the approach of the exploration and visualization of these sites to society without the need of performing underwater immersions.
The IEO has technological capabilities to deal with this type of work.
Fusion of optical and acoustic data to generate 3D models is suitable for the study of underwater shipwrecks.The ship is placed in navigation position.The stern is the part most buried in the sand in such a way that its propellers cannot be seen.In this area there is a hatch that leads to two interior spaces.At the moment the shape of the boat and its imposing prow are perfectly appreciated.Thanks to these results, the causes of the shipwreck can be checked on the 3D model.Acoustic and photogrammetric reconstructions allow the study of those parts of the ship that have not been damaged and do accuracy measurements.
This study shows that it is possible to apply color and contrast enhancement filters designed for underwater images to 3D point clouds using a synthetic image, improving significantly the results.
The methodology applied to the process of both acoustic and optical point clouds is highly accurate, demonstrating that it is possible to carry out the subsequent fusion of the data with a simple tool.This merging process keeps the advantages of both types of marine exploration technology in a single data set.
This innovative combination of very high resolution acoustic data with optical data is a methodology that can be extrapolated to another series of non-invasive underwater studies, such as vulnerable benthic habitats.

Figure 1 :
Figure 1: Río Miera cargo ship (source: sitepecios.com)On December 6, 1951; a small merchant called Mogador collided head on Río Miera and the cargo began to sink quickly.Cantabrian coast was the scene of many shipwrecks, and some of them remain without explore.Nowadays Río Miera is one of the reference points of immersion of the Cantabrian coast.

Figure 3 :
Figure 3: 3D reconstruction of video transect in Pix4D Mapper Pro software -Green points are camera position

Figure 4 :
Figure 4: Original video-frame recorded in cruise It is necessary some kind of improvement to the images acquired from the application of different filters or histogram adjustments that allow reconstructing the real color and avoiding the loss of contrast.

Figure 5 :
Figure 5: Synthetic image with 3D point cloud RGB values, having lost the spatial component of each point.

Figure 6 :
Figure 6: Acoustic data point cloud on processing

Figure 7 :
Figure 7: An example of video-frame before (up) and after (down) of LAB + ACE filters applied

Figure 8 :
Figure 8: 3D point clouds before (down) and after (up) enhancement.A fictitious geometric displacement in z values has been introduced for the correct visualization of the two datasets.

Figure 10 :
Figure 10: Video-frame with some fine and vertical elements in the structure of shipwreckOnce the fine adjustment has been made, the point cloud can be transformed in a single data set in which both sources of optical and acoustic data are integrated (Fig.11).

Figure 11 :
Figure 11: Merging optic and acoustic 3D point cloud using ICP algorithm in CloudCompare software

Table 1 .
Basic characteristics of dataset processing in Pix4D The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W10, 2019 Underwater 3D Recording and Modelling "A Tool for Modern Applications and CH Recording", 2-3 May 2019, Limassol, Cyprus

Table 2 :
Results from ICP fine registration