DIVERS-OPERATED UNDERWATER PHOTOGRAMMETRY: APPLICATIONS IN THE STUDY OF ANTARCTIC BENTHOS

Ecological studies about marine benthic communities received a major leap from the application of a variety of non-destructive sampling and mapping techniques based on underwater image and video recording. The well-established scientific diving practice consists in the acquisition of single path or ‘round-trip’ over elongated transects, with the imaging device oriented in a nadir looking direction. As it may be expected, the application of automatic image processing procedures to data not specifically acquired for 3D modelling can be risky, especially if proper tools for assessing the quality of the produced results are not employed. This paper, born from an international cooperation, focuses on this topic, which is of great interest for ecological and monitoring benthic studies in Antarctica. Several video footages recorded from different scientific teams in different years are processed with an automatic photogrammetric procedure and salient statistical features are reported to critically analyse the derived results. As expected, the inclusion of oblique images from additional lateral strips may improve the expected accuracy in the object space, without altering too much the current video recording practices.


INTRODUCTION
Recently, ecological studies focusing on marine benthic communities have received a breakthrough from the application of a variety of non-destructive sampling and mapping techniques.These improvements have been catalysed by the technical advances reached in underwater image recording, by the availability of 3D reconstructions techniques and of a variety of software for ecological analyses of photos and videos depicting benthic assemblages.An important innovation was represented by the introduction of photographic or video samplings, non-invasive techniques (Joiner, 2001) which brought a number of advantages, namely: i) creation of permanent records, ii) reduction of time spent in the field, iii) more objective interpretation of the collected data, allowing analyses less prone to human errors and personal interpretation, iv) preservation of the marine habitat.In the last years, the availability of affordable (and often also open-source) software to process and analyse the images have also pushed the popularity of 3D image-based modelling approaches.Overall, these techniques provide low-impact and cost-effective methodologies that enable the observation of natural processes with the lowest possible disturbance on communities and organisms coupled with a higher efficiency and methodological reliability in the analyses.Due to these reasons, 3D imaging techniques have been considered as suitable tools in the scientific community to study fragile, slow growing and potentially endangered environments, such as coral reefs or Antarctic benthic communities.However, to train people from different application fields to properly record the images best suited for 3D modelling of underwater environments is still an open issue.

Paper contribution
In this contribution, we present an investigation on diveroperated photogrammetry for the description of shallow-water rocky-bottom benthic communities in Antarctica.The main purpose of this interdisciplinary study is to critically present a processing procedure for the exploitation of video footage recorded in past campaigns according to scientific diving practice, which usually consist in single or multiple strips along transects.The video streams were collected in three different campaigns, in 2006, 2015 and 2017, by two different international teams from New Zealand (NZ) and Italy.A single revisit was carried out in 2017 by NZ researchers at the 2015 surveyed site, producing a temporal replicate of one of the transects.The video footage from 2006 and 2015 were not recorded with the specific aim of retrieving 3D information, consequently requiring a cautious processing pipeline to derive 3D metric information of the underwater sites.On the other hand, as results of the ongoing international collaboration, the videos from 2017 were recorded implementing for the first time a new camera network configuration which includes oblique views.

Trends
Benthic ecologists' interest for visual observation of seafloor and inhabiting organisms has a long history, as long as the interest for benthic communities' structure and processes (Aleem, 1956).As natural evolution of sampling methods, scientific divers have moved from simple counting along transects to the use of photographs and video footage with the aim of simplifying the data acquisition, at the same time exploiting methods of digital recording not affected by operators subjectivity and not altering the marine habitat.
There is a wide variety of available methodologies to quantitatively study benthos (Hill et al., 2004), among which non-destructive approaches assure environmental and habitat preservation (Parravicini et al., 2009;Mallet and Pelletier, 2014).Non-destructive visual-based methods include: underwater visual census techniques (UVC) at shallow waters (Leonard and Clark, 1993;Langlois et al, 2010); remote underwater video techniques (RUV) in deeper waters (Williams et al. 2015); and other approaches of analysis on sampled imagery (Dumas et al., 2009;Teixidò et al., 2011;Kipson et al., 2011).The next step forward has consisted in employing automatic computer-vision and photogrammetric methods for processing the acquired underwater video footages and producing typical photogrammetric output (i.e.orthophoto mosaics, 3D point clouds, digital surface models).Compared to traditional methods used in the scientific diving community, the advantages are enormous, allowing also for objective postprocessing analyses (i.e.not in situ diver-operated estimations) with benthic image analysis softwares (e.g.Seascape, Teixidò et al. 2011, andphotoQuad, Trygonis andSini, 2012).Underwater photogrammetry has then been exploited in hostile environments or remote areas (Westoby et al., 2012), in applications where accuracy, high level of details and minimum environmental impact are fundamental, such as, for example, in coral reef (Burns et al., 2015a;2015b;Annelli et al., 2017) and seafloor mapping and monitoring (Friedman et al., 2012;Guo et al., 2016).

Needs
In the analysis of benthic communities, identification and counting of specimens are key operations, which may benefit from the availability of texturised 3D models of the environment.The possibility of exploiting the combination of geometric and radiometric (color) features allows for a more accurate and reliable classification of image samples.For example, in the case studies presented in this paper, the most abundant species in the benthic community is a sea urchin, named Sterechinus neumayeri.Specimens of this species are characterized by a well-defined spheroid shape of size usually ranging from a minimum diameter of about 30 mm to a maximum of 70 mm at an age of about 40 yr (Brey et al., 1995), showing a purple prevalent colorization of the body.Based on these well recognizable features, it should be not difficult to detect Sterechinus-like shapes in the model and assign them to the corresponding species or group.Another relevant task for understanding benthic dynamics and changes is to estimate growth or changes in time of sessile benthic organisms, which can highly differ from species to species.In some cases, such as, for example, corals in tropical areas, the growth rate can be in the sub-centimeter range (Bessat and Buigues, 2001).For massive sessile organisms (i.e.organisms with voluminous body that dwell fasten to the seabed), such as the large sponges dwelling in the seafloor of Tethys Bay, which represent keys species of Antarctic benthos, the required scale can be an order of magnitude higher, especially when the specimen volume estimation is of particular interest.The latter analysis is only possible by non-destructive sampling methods that allow a volume's estimation without physically removing the samples.In the first case (i.e.corals in tropical areas), global changes in a vast site are to be estimated, which requires highly accurate multi-temporal measurements.Geo-referencing in the order of millimetre level is crucial, and deformations of the photogrammetric model are to be kept as low as possible, to allow the detection of sub-centimetric variations.This principally requires (i) the establishment of an accurate and robust network of underwater ground control points, (ii) the execution of a reliable image acquisition configuration, (iii) the adoption of a rigorous mathematical model for a proper formulation of the image formation process through the water.
The image acquisition task can be facilitated with a proper planning and the adoption of supporting devices to indicate, for example, the path to follow, the depth to keep, etc. (Menna et al., 2013).On the contrary, to set and measure a stable control network with sufficient accuracy is a requirement very hardly met underwater (Capra et al., 2017;Skarlatos et al., 2017).Due to several factors, including different sensors of digital cameras (i.e., global or rolling shutter), lens quality, poor mechanical stability of digital cameras/lenses, images are far from being an ideal central perspective representations of the observed scene.A mathematical formulation to rigorously model the image formation process in water would be certainly beneficial; however, a robust self-calibration approach may already provide sufficient accuracy for a number of applications (Shortis, 2015).This is no longer valid when the camera network geometry is weak, as in case of elongated strips with only nadir images without the availability of ground control points.In such unfortunate cases, systematic errors may accumulate, producing a bending in the photogrammetric model (Menna et al, 2018;Nocerino et al., 2014).
When the aim of the 3D modelling project is the inspection of massive sessile organisms, the requirements are less restrictive, whereas to guarantee good image quality, robust camera network configuration, and reliable scaling method still represent critical issues.
Another promising application field is the study of the bottom topographic features at a local and fine scale (Friedman et al., 2012;Storlazzi et al., 2016) that could provide new insight on the correlation between these features and organisms distribution.These analyses necessitate the combined use of specific tools of benthic image-analysis (e.g.GIS tools for raster spatial analysis) coupled with photogrammetric tools to estimate bottom rugosity and microrelief in rocky bottom.Employing a similar non-destructive sampling method allows to record and geolocate the actual position of organisms and to correlate it with three-dimensional features of the substrate at a very fine scale.Such a result could be only barely and costly achieved by using other sampling methods.Still, critical is the achievement of an accurate geo-referencing of the 3D models.

THE ANTARCTICA ENVIRONMENT
In many underwater studies, images are acquired by SCUBAdivers, according to pre-defined protocols.Tropical areas, characterised by mild-water temperatures and shallow-depth sites, which allow for long diving times, are ideal for diversoperated photogrammetry.
In harsh environments, such as polar areas, where seawater temperatures are below zero and close to the seawater freezing point (i.e.-1.8°C), a careful planning of the image acquisition strategy is mandatory to guarantee efficiency and rapidity, minimising the diving time and saving costs.Antarctica diving operations are even more complex, being performed under the pack-ice, usually during the first half of austral summer (Fig. 1) under strict safety protocols.The ice coverage represents, on one hand, a supplementary logistic obstacle but, on the other hand, ensures the best available environmental conditions, since, under the ice, water is crystal clear, due to the absence of particulate or suspended matter or organisms (pre-bloom conditions).
A number of studies confirm that Antarctic sea waters have significantly different bio-optical properties in comparison with temperate latitude, especially in terms of a reduced level of backscattering due to the low presence of particulate (Dierssen and Smith, 2000).Moreover, pack-ice also guarantees the attenuation of swells, as, in presence of sea-ice coverage, water column results less exposed to waves and currents and thus calmer, a condition very advantageous for divers to keep a neutral buoyancy.Due to the challenging field condition, and despite being the elective method for shallow rocky bottom (Brueggeman, 2003), as well as in other latitudes, SCUBA-operated studies in Antarctic environment are not so common (Cummings et al., 2006;Trush et al. 2010)-with only one example of long lasting observation study in coastal areas (Dayton et al. 2016)-, as well as ROV-operated photo and video sampling-based studies (Gutt and Starmans, 1998;Teixidò et al. 2002Teixidò et al. , 2006)).Until now, there are no published studies reporting video-based SCUBAoperated samplings with photogrammetric application on Antarctic benthos spatial analysis.Finally, one must bear in mind that underwater operations, in particular in harsh environment, should be kept as simple as possible to limit bottom time and reduce the risks for the divers.For this reason, the collection of ground control points, through for example trilateration methods as done in underwater archaeology, is neither effective nor safe in the Antarctica environment.
Figure 1.The study area, Tethys Bay (Ross Sea, Antarctica) during the first half of December with ice-pack still present.

The area under investigation
The coastal area of interest (Fig. 1) is located in the 2 Km-wide Tethys Bay (Ross Sea, Antarctica), which is characterized by granite steep cliffs and a seafloor with a mild slope settled by gigantic boulders, intermixed to coarse sand and gravel.This area is covered by the annual sea-ice layer until mid-December, when field sampling activities of the close Italian Antarctic coastal research base (Mario Zucchelli) usually start.Thus, especially diver-operated activities have to face this logistic constraint: in order to allow divers access the water, holes are drilled in the ice layer, often almost 3 meters-thick, by icedrillers and a diving camp is installed on the pack near the holes (Fig. 2).

The SCUBA-operated video-sampling
Video-recording of the underwater transects is the final operation in the sequence of field work activities, which require the drilling of the pack-ice, arrangement of the diving-camp, preparatory dives for testing the underwater equipment and exploring the site of interest.In both 2006 and 2015 campaigns, the underwater recording was carried out following the same procedure.Transects' paths were marked with a 20 m long meter tape laid between heavy body (e.g.stainless steel stakes or concrete blocks).The meter tape was well outstretched and showed clearly readable scale ticks in several segments of its total length.As shown in the next sections, these segments were employed to fix the scale of the photogrammetrically derived models.Moreover, in both campaigns, transects were recorded with the camcorders at fixed distance from the seafloor (about 1 m above the bottom).
A more recent video-recording survey was carried out in 2017 in order to create temporal replicates of one of the 2015 transects with the aim to estimate changes after 2 years.During 2017 operations, NZ divers followed the same transect lines defined in 2015, by orienting their path thanks to the GPS coordinates of starting, midway and ending points of the transect and the presence of heavy fixed body as markers for these points.This revisiting procedure confirmed the accessibility and availability of the permanent sites chosen in 2015, a fundamental condition for a monitoring program and, at the same time, an arduous task in such a remote and inhospitable place, as Antarctic ice-covered coastal sites.

RECOVERING 3D INFO FROM DIVER-OPERATED VIDEO STREAMS
Videos produced during under-ice dives were recorded by using two different Sony camcorders, i.e. the Sony HVR-HD1000E and SONY HDR-HC7 respectively in 2006 and 2015 campaigns).In both cases the video format was HDV1080i, with a frame rate of 25 fps.The recorded videos have a duration ranging from about two minutes up to five minutes (three and a half minutes in average).Six different test sites were recorded, with lengths varying from 16 m to 20 m.The camcorders were placed in underwater housings equipped with flat ports and external lights.During the acquisition, the recording device was mainly oriented in a nadir looking direction, i.e. pointing orthogonally towards the seafloor, as in the common practice of SCUBA-operated photographic or video sampling.The focal lengths were fixed at the nominal value of 4.5 mm and 7.5 mm respectively for the two campaigns.In 2017, divers repeated the same video-sampling procedures using a mirrorless digital camera, a Sony A7IIs, equipped with a Sony FE 16-35mm f/4 ZA OSS Vario-Tessar T* Lens (fixed at 16mm),housed in a Nauticam Housing NA-A7II with a dome port, and equippedwith a couple of LED video dive light (Keldan 4X Compact).

Photogrammetric processing
The original video streams from 2006 and 2015 were processed according to the following steps: 1. the analogue video were digitized with the Sony frame grabber (model DVBK2000E); 2. the frames were de-interlaced and extracted at the full frame rate with the VideoLAN software (VLC).Videos from the Sony A7IIs were processed in ffmpeg (www.ffmpeg.org)by extracting only the keyframes.A mixed procedure of image orientation and self-calibrating bundle adjustment was done using the commercial software Agisoft Photoscan and the open source damped bundle adjustment toolbox DBAT (Börlin and Grussenmeyer, 2013) v0.7.2.0 for MATLAB environment.The use of DBAT software allowed to reprocess the image observations automatically extracted from Photoscan thus providing a more advanced tool for the analysis and revision of the bundle adjustment results.Through for example the analysis of the full variance-covariance and correlation matrices it was possible to understand the precision of self-calibrated parameters, their significance and dangerous correlations between interior and exterior orientation parameters.With the aim of highlighting some critical issues of the photogrammetric processing, the video from 2017 (T1-2017) was processed twice: i) single nadir strip configuration and ii) together with two additional oblique looking strips, in order to emphasize the benefit of multiple strips on the global accuracy of 3D measurements.All the datasets were processed with a minimally constrained self-calibrating BA.Six degrees of freedom (DOF) needed for datum definition were solved by fixing the six exterior orientation parameters (three translations and three rotations) of the camera located nearest to the barycentre of the camera network.An average scale factor was on multiple segments of the ruler visible in the transects.

Self-calibrating BA
Table 1 summarises the main statistics from the self-calibrating BA.The processing of the video footage from 2006 and 2015 shows the highest RMS of image residuals and σ0, while the dataset T1-2017 with only nadir images provides the highest standard deviation value of the principal distance (focal length).In Figure 3, the camera network and 3D tie points are reported for all the processed dataset.The dataset S1-2006 shows a light bending of the photogrammetric model (Fig. 3.a), deformation that is much more critical for the T1-2017 only nadir (Fig. 3c).The bending is significantly reduced when the oblique images are included in the BA, as shown in Figure 3 The trend suggests a bending toward the extremities where the standard deviation can be as high as 15 mm (Fig. 4.a), three times bigger than the corresponding values for the camera network with the oblique views.In this case, the highest standard deviations are associated to points external to main transect, triangulated with a small angle (Fig. 4.c).

3D models analyses and comparisons
3D models of the seabed and of large organisms (Fig. 5) were derived from the T2-2015 and T2-2017 and employed for different spatial analyses, such as the counting of invertebrates, and measurements of shape and volume of large organisms such as sponges.Comparison of different temporal epochs was possible only from 2015 transects, that were revisited in 2017 allowing to estimate changes in volumes and organisms' rate growth.Preliminary analyses on the first of these transects showed a high (and totally unexpected) dynamism in some species.In particular, it is evident a notable difference in sponges population, with a number of specimens completely disappeared in two years.The bigger was a specimen of Mycale acerata, having a volume of about 0.03 m 3 in first video-sampling of 2015, and completely absent in the 2017 revisit (Fig. 5.a).

Statistical ecological analysis
Geospatial packages of R (R-project.org) and GIS (e.g.QGIS, qgis.org)software and specific image analysis software, such as those based on chromatic hierarchical segmentation processes (e.g.Seascape, Teixidò et al. 2011, andphotoQuad, Trigonis &Sini, 2012), allow to obtain quantitative measures useful to document the changes occurring in the benthos and hence establish a monitoring program of the communities studied.A first ecological outcome of the image analysis procedures concerns spatial patterns of S. neumayeri, that thanks to the application of appropriate indices (K-Ripley index, Ripley, 1976, Fig. 5) resulted to be clustered.It is important to highlight that, from an ecological point of view, the possibility to quantitatively describe this kind of phenomena (probably already studied and described, but only qualitatively), it's an original and relevant outgrowth allowed by photogrammetric output (e.g.3D models or orthophotos) that can be spatially georeferenced and analized by GIS platform tools.

CONCLUSIONS AND OUTLOOK
This paper discussed the preliminary results on an ongoing international collaboration aiming at establishing a transitional procedure to exploit available video footage recorded according to scientific diving practice (from past campaigns) and developing, at same time, best practices for 3D digital recording of underwater environment under harsh conditions.The impossibility of establishing an accurate and reliable geodetic underwater network in Antarctica is pushing us to identify best practices in video recording, along with the identification of high-performance underwater imaging systems (global shutter, fixed focal length, fixed focus, use of a dome port).Moreover, the development of specific open source software with extended mathematical modelling of multimedia photogrammetry is also advisable.The inclusion of oblique images from the two lateral strips has proved to ameliorate the results of the self-calibrating BA, thanks to a better estimate of interior orientation parameters of video transects not specifically designed for photogrammetry.
It is worth noticing that values such as point residual RMS or σ0 are necessarily able to highlight issues in the processing, such as bending of the 3D models.3D point precision vector lengths can provide better clues about the quality of the photogrammetrically derived 3D models, even though it is not possible to estimate the actual accuracy degradation.
Owing to the lack of ground references, it is not possible to state the absolute accuracy of the photogrammetrically derived 3D models, making them not suitable for detecting global changes in the sub centimetric level.However, the 3D models and derived orthoimages can be exploited for local analysis (sections 5.2 and 5.3).The combination of photogrammetry and spatial analytic software used in ecology, thus, represent a promising tool in monitoring activities in Antarctica, that may lead our analytical capabilities in benthos patterns and dynamics study to an unprecedented level of detail.

Figure 2 .
Figure 2. SCUBA-operated field activities in ice-coverage conditions: a hole is produced by an ice drillers in the pack ice (A), showing the thickness of ice coverage (about 3 m) (B).Through these holes (C) SCUBA-divers, adequately equipped, can access to -1.8°C cold polar waters (D).
Figure 4. 3D tie points colored according to: a) the precision vector length (in meters) for T1-2017 with only nadir looking images, b) the precision vector length (in meters) for T1-2017 with nadir and oblique looking images; c) the intersection angle (in degrees) for T1-2017 with nadir and oblique looking images.

Table 1 .
Metadata, self-calibrating BA results and statistics of the processed video streams