SEAFLOOR MAPPING FROM MULTISPECTRAL MULTIBEAM ACOUSTIC DATA AT THE EUROPEAN OPEN SCIENCE CLOUD

Abstract. Recent technological advances in the underwater sensing instrumentation provide currently active multibeam echosounders that can acquire backscatter observations from multiple spectral frequencies. In this paper, the main objective was to design, develop and validate an efficient and robust multispectral, multibeam data processing framework including advanced machine learning tools for seabed classification. In order to do so, we have integrated different machine learning tools like support vector machines and random forests towards the classification of seabed classes. We have performed extensive experiments with different splitting ratios, regarding training and testing sets, in order to assess possible overfitting. The entire pipeline has been implemented in a scalable containerized manner in order to be deployed in cloud infrastructures and more specifically at the European Open Science Cloud. Experimental results, the performed qualitative and quantitative evaluation along with the comparison with the state of the art indicated the quite promising potential of our approach.



INTRODUCTION
Accurate seabed mapping is of significant importance for numerous marine and coastal applications. Seabed thematic classes, type of sediments and seafloor materials determine the turbidity of water, provide a substrate for marine benthic organisms, host organic matter and are involved in biogeochemical exchanges. Moreover, monitoring their dynamic changes is also crucial since for example the redistribution of sediments in large geographical scales due to hydrodynamical processes has direct implications for geological basin/ coastal evolution. Therefore, efficient seabed mapping as well as detection of seabed composition changes through time are increasingly important and currently required by marine scientists and stakeholders underpinning decision making in relation to marine spatial planning and marine protected regions design and policy (Diesing et al., 2016).
In order to do so for large geographical scales acquiring images from any manned or robotic marine mapping system (ROV, AUVs, USV, etc) is not currently as effective as scanning seabed with acoustic echosounders. However, the standard multibeam systems are collecting backscatter data at a single frequency or at a narrow band around the central, monochromatic, frequency (Clarke, 2015). In the last two decades, in order to satisfy the need for more extensive exploration of the seabed, several applications have been developed using multibeam systems that acquire repeatedly and simultaneously the same line in multiple frequencies (Gaida et al., 2018). Observing at multiple frequencies the reflectance of seabed materials can leverage the applicability of cutting-edge multispectral multibeam systems like the ones that provided the benchmark datasets for the 2017 R2Sonic multispectral multibeam contest.
Although these novel multibeam echosounders allow the acquisition of spatially and temporarily co-registered multispectral backscatter data, the full exploitation of these type of multispectral multibeam data is challenging. Apart from bathymetric mapping, the backscatter information at different spectral regions can be employed for tackling various seafloor mapping and classification tasks.
Towards this end, a number of recent studies aimed at exploiting multispectral multibeam data and classify the seabed. Costa (2018) employed boosted regression trees for the classification in three thematic classes combining multispectral, topography and geographic analysis information. Quantitative results indicating an overall accuracy (OA) 96,0% and Kappa coefficient at 82,1%. Moreover, Buscombe and Grams (2018) targeted four and seven seabed classes depending on the dataset. They employed a Gaussian Mixture Model and Conditional Random Field classification method, achieving OA ranging between 75% to 84%. Gaida et al. (2018) classified the multispectral data at nine (and four) not-thematic but spectral classes based on a Naïves Bayes classifier. Results were validated through a statistical calculation based on a Bayesian method.
In a similar unsupervised manner, Campbell et al. (2018) targeted eight spectral not-thematic classes based on a Gaussian maximum likelihood classification. A canonical variate analysis approach was used for validation purposes. Brown and Varma (2018) combining multispectral layers and topography (in terms of bathymetry, slope and curvature) information in order to classify the seabed in nine classes based on a HyperCube Segmentation method. Standard deviation errors between the classified map and ground truth were examined for validating the derived results.
Towards a similar direction, in this paper we present a methodology for the efficient and accurate seabed classification based on multispectral multibeam data. Our experiments were concentrated on Bedford Basin and Patricia Bay datasets (provided by the R2Sonic Challenge) consisting of multispectral multifrequency data at 100 kHz, 200 kHz and 400 kHz. The designed and developed methodology included a pre-processing and bathymetry estimation step, the reference data construction and the application of the classification procedure as well as the qualitative and quantitative evaluation. All software modules have been implemented in a scalable containerised form in order to be deployed in the European Open Science Cloud in the framework of NEANIAS EU project.

Datasets and Study Areas
In this section three multispectral multibeam datasets from the R2Sonic Multispectral Challenge 2017 are presented. They are consisted of three backscatter responses at different wavelengths i.e., 100, 200 and 400 kHz respectively.
From the provided point clouds three georeferenced images were derived corresponding to each spectral backscatter layer. These layers were stacked together for the following processing steps as well as for visualization purposes. The present study was focused on two areas: Bedford Basin, Halifax, Nova Scotia (1,83 km 2 ), with images acquired in March 2016 and May 2017 and water depth 10 to 83m (Figures 1 and 2) and Patricia Bay (0,94 km 2 ) with images acquired in November 2016 and water depths ranging from 15 to 70m ( Figure 3).

Methodology
The main goal of the designed and developed methodology was to process at a highly automated manner the initial multispectral multibeam datasets and produce highly accurate seabed maps. To this end, the aforementioned datasets were employed and from the initial point clouds data the bathymetry was derived for each area (Bedford Basin and Patricia Bay). Bathymetry was conducted via interpolation methods by retaining at every cell the minimum value (lowest value). Moreover, georeferenced images were computed and employed for the classification of the seabed.  Overall, the developed approach is consisted of the following steps: a. Multispectral multibeam data pre-Processing b. Bathymetry mapping c. Reference data construction/ splitting ratios d. Classification procedure e. Quantitative and qualitative validation

Pre-processing and Bathymetry Mapping
This processing step is quite important since the point cloud comes occasionally with noise and/or artifacts locally or at the entire data/ image domain. Noise was detected automatically based on an algorithmic procedure with Gabor filters calculation. The first dataset (i.e., Bedford Basin 2016) had significant noise levels mainly due to overlapping areas among the numerous scan lines. The noise was tackled by both interpolation and standard inpainting correction methods.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition)

Reference/ Ground Truth Data Construction
In this section, the procedure for the construction of the reference data is briefly presented. For Bedford Basin reference data were constructed by annotating in the image domain polygons based on the available in-situ data that were collected during both surveys in 2016 and 2017. For Patricia Bay, due to the lack of in-situ data, geological maps (Benjamin R. Biffard, 2003) were employed and combined with the backscattered imagery and bathymetry.

Classification Procedure
Apart from the three multispectral backscatter layers the bathymetry layer was also employed for the classification procedure. Moreover, in order to examine possible overfitting, different splitting ratios regarding the training and testing sets were composed. Two main classifiers were employed i.e., Random Forest (RF) and Support Vector Machines (SVMs).

Quantitative and Qualitative Validation
The validation of the classification maps was performed based on standard quantitative metrics. In particular, all confusion matrices were calculated and examined. Moreover, the standard metrics of Overall Accuracy (OA), User's Accuracy (UA), Producer's Accuracy (PA) and Kappa coefficient (Kappa) was also computed, compared and discussed for every case.

EXPERIMENTAL RESULTS AND VALIDATION
In this section, the resulting after the application of the developed methodology, classification maps are presented along with their quantitative and qualitative validation for every study area i.e., Bedford Basin 2016, Bedford Basin 2017 and Patricia Bay 2016 (Section 3.1). Moreover, in order to examine how robust and generic is the proposed approach in terms of model overfitting we have conducted numerous experiments with different splitting ratios regarding the training and testing sets (Section 3.2). Last but not least, our results are compared with the current state-of-the-art algorithms most of which have been presented in the framework of the 2017 R2Sonic Multispectral Multibeam Challenge (Section 3.3).

Evaluation of classification results per study area
In this Section results based on the Random Forest classifier are presented. In all cases the input data were the backscatter georeferenced images (100, 200 and 400 kHz) and bathymetry. Moreover, results were derived after the random separation of the reference data into training and testing sets.
In all cases, results are presented after employing 30% for training and 70% for validation. The quantitative validation was based on the standard metrics of user accuracy (UA), producer accuracy (PA), overall accuracy (OA) and Kappa index. For all presented confusion matrices, columns represent the classification labels and rows represent the testing data. Values are given in pixels except PA, UA, OA and Kappa. The diagonal values express the PA for each class.
The classified seabed at Bedford Basin 2016 is presented at Figure 4. Five thematic classes were considered based on the available reference data namely, (i) Yellow Sand, (ii) Sand and Algae, (iii) Mud Sand and Corals, (iv) Sand and (v) Gravel. In particular, the class Yellow Sand is covering a quire large proportion of the region.
Sand and Algae was mainly identified on the central and northern part of the region, while Mud Sand and Corals mainly in the central-southern part of Bedford Basin. The quantitative evaluation indicated a relatively high OA of 99,6% (Table 1).
In particular, both PA and UA for every class were above 99%. Among the misclassification cases was that 674 pixels out of 183959 of Sand testing pixels were erroneously labelled as Mud Sand; and 241 pixels as Gravel. The class with the relatively lowest PA performance was Sand and Algae.
Qualitatively, by comparing the combined backscatter and the produced classification map (Figure 4) certain misclassifications cases are observed in the south-western part of the region with a number of pixels labelled as Sand and Algae instead of the correct Mud Sand and Corals ( Figure 5).  (Table 1) indicated an overall accuracy rate of 99,6%. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) Figure 5: Indicative misclassification cases at Bedford Basin 2016 as identified during the qualitative evaluation assessment.
The classified seabed at Bedford Basin 2017 is presented at Figure 6. Five thematic classes were considered based on the available reference data namely (i) Yellow Sand, (ii) Sand and Algae, (iii) Mud Sand and Corals, (iv) Sand and (v) Gravel.
In particular, the class Sand and Algae is covering a quire large proportion of the region. Yellow Sand was mainly identified on the southern part of the region, while Sand mainly in the northwestern part of Bedford Basin and in the perimeter of the Gravel class. The quantitative evaluation indicated a relatively high OA of 99,9% (Table 2).
Analysing Table 2, both PA and UA for every class were above 99%. Among the misclassification cases was that 545 pixels out of 369580 of Sand testing pixels were erroneously labelled as Mud Sand and Corals. The class with the lowest PA performance was Sand and Algae. Qualitatively, by comparing the combined backscatter and the produced classification map (Figure 6), no crucial misclassifications were detected.
The main differences between the classified seabed of Bedford Basin 2016 and 2017 are presented at Figure 7. The blue one, represents the Sand class (south-western part), which was increased spatially during 2017. In addition, the other two frames (black and red) represent the increase in coverage of Sand and Algae Class in 2017.  (Table 2) indicated an overall accuracy rate of 99,9%. The classified seabed at Patricia Bay is presented at Figure 8. Three thematic classes were considered based on the available reference data namely (i) Sand, (ii) Mud Sand and (iii) Gravel.
In particular, both Sand in the northern part and Mud Sand in the southern, are covering a quire large proportion of the region.
Gravel was mainly detected on the boundaries of the southern and northern part of the region. As for the quantitative evaluation indicated a relatively high OA of 98,8% (Table 3).    (Table 3) indicated an overall accuracy rate of 98,8%.
In Table 3, both PA and UA for every class were above 95%. Among the misclassification cases was that 399 pixels out of 600883 of Gravel testing pixels were erroneously labelled as Mud Sand. The class with the lowest PA performance was Mud Sand. Qualitatively, by comparing the combined backscatter and the produced classification map (Figure 8), one main misclassification case was detected in the south-western part of the region with a number of pixels grouped as Sand instead of Mud Sand (Figure 9).

Experiments with Training/ Testing Splitting Rations, Overfitting
In this Section several experimental results with different splitting ratios, based on the Random Forest and SVM Linear classifier are presented in Table 4. In all cases the input data were the backscatter georeferenced images (100, 200 and 400 kHz) and bathymetry.
The quantitative validation was based on the overall accuracy (OA) and Kappa index. With bold numbers are the higher values between the SVM and RF experiments.  In all cases RF was assessed as the most proper classifier for every study area regarding the above results. In particular, concerning Bedford Basin 2016 region, the best experimental result was achieved using 30% of the Dataset as Training. In addition, for the Bedford Basin 2017 and Patricia Bay, the highest performance results were achieved using 70, 50 and 30% ratios for training.

Comparison with the State of the Art
In this section, our results were compared with the current state of the art algorithms/methods which were conducted mainly in the framework of the 2017 R2Sonic Multispectral Multibeam Challenge are presented (Table 5).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) In particular, the most crucial variations between the present study and the state of the art, are expressed in Table 5 indicating the study area, the number of seabed classes, the classification method and when available the results from the quantitative evaluation. In particular, a number of studies in the same datasets have not presented confusion matrices along with the corresponding PA, UA, OA and Kappa quantitative results. Those studies presented mainly a qualitative assessment and therefore no quantitative information was available. Buscombe and Grams (2018) have employed GMM and CRF models for the classification task. The reported OA was around 84% and 75% for Bedford and Patricia study areas, respectively. Costa (2018) employed boosted regression trees as a classifier and discriminated three seabed classes with a reported OA at 96%. Our methodology in all cases outperforms previous research efforts.

CONCLUSIONS
In this paper, a methodology for the efficient classification of the seabed based on multispectral multibeam data has been designed, developed and validated. In particular, we have integrated different machine learning classifiers and other software modules for data preprocessing and validation. We have performed extensive experiments with different splitting ratios, regarding training and testing sets, in order to assess possible overfitting. The entire pipeline has been implemented in a scalable containerized manner in order to be deployed in cloud infrastructures and more specifically at the European Open Science Cloud.