QUASI-ORTHORECTIFIED PROJECTION FOR THE MEASUREMENT OF RED GORGONIAN COLONIES

This study presents a practical method to estimate dimensions of Paramuricea clavata colonies using generic photographic datasets collected across wide areas. Paramuricea clavata is a non-rigid, tree-like octocoral; this morphology greatly affects the quality of the sea fans multi-view stereo matching reconstruction, resulting in hazy and incoherent clouds, full of ”false” points with random orientation. Therefore, the standard procedure to take measurements over a reconstructed textured surface in 3D space is impractical. Our method overcomes this problem by using quasi-orthorectified images, produced by projecting registered photos on the plane that best fits the point cloud of the colony. The assessments of the measures collected have been performed comparing ground truth data set and time series images of the same set of colonies. The measurement errors fall below the requirements for this type of ecological observations. Compared to previous works, the presented method does not require a detailed reconstruction of individual colonies, but relies on a global multi-view stereo reconstruction performed through a comprehensive photographic coverage of the area of interest, using a lowcost pre-calibrated camera. This approach drastically reduces the time spent working on the field, helping practitioners and scientists in improving efficiency and accuracy in their monitoring plans.


INTRODUCTION
Climate change is threatening underwater habitats worldwide.The structural complexity of the benthic communities is compromised where impacts involve engineer species on soft and hard sea bottom at any latitude (Sánchez, 2015).Ecosystem functions are enhanced by the presence of species with three-dimensional growth (Cerrano et al., 2010) .The study of the growth, abundance, and recovery capability of those species provides valuable information on the conservation status of the entire habitat.
In the Mediterranean Sea, the fans of the red gorgonian Paramuricea clavata (Risso, 1826) create complex seascapes and contribute to one of the richest temperate biogenic habitat: the coralligenous (Ballesteros, 2006).The intricate structure of the sea fans provide a shelter for many species at different life stages, attenuate the incidence of light and facilitate the growth of coralline algae.As result, the red gorgonian is considered an "engineer species", as its growth helps building the structure of the seascape (Valisano et al., 2016).In the last decades, P. clavata has been subjected to important episodes of mass mortality that have caused a general reduction of its distribution on the upper bathymetric limit, and local extinctions across the Mediterranean Sea (Linares et al., 2007).The events have been blamed in part on declining water quality and elevated water temperatures correlated to changes of global climate (Vezzulli et al., 2010).Therefore, to develop conservation actions aiming at protecting coralligenous habitats, we require a comprehensive understanding of the resilience of the species.
In recent years, the improvement of tools for dense photogrammetry suggested their use in the investigation of marine habitat.
Drap (Drap et al., 2013) used Remote Operated Vehicles (ROV) to study the growth of Corallium rubrum in-situ.Palma (Palma et al., 2017) generated high resolution orthomosaics of wide areas (2000 m 2 ) of coral reefs using a semi-autonomous underwater vehicle equipped with low budget cameras.Pizzarro (Pizarro et al., 2017) proposed an effective data sampling method for investigating coral reefs based on a single diver effort.In (Gutierrez-Heredia L, 2016), authors created highly detailed 3D models of submerged and dry corals using low cost monocular cameras and then compared them.Different issues can be addressed using a detailed 3D reconstruction of the seabed, including the automatic classification of benthic communities, the spatial mapping of populations, the estimation of the biomass (Thornton et al., 2016), and the temporal monitoring of the progress and regression fronts of colonies due to natural or extraordinary phenomena.In this paper we will address the last issue, the monitoring, by presenting a method for measuring P. clavata colonies.
Classic approaches for coral measurements involve manual recording of measures on site by a diver.These methods require the presence of experienced divers and are very time-consuming.The presence of corals determines what is called terrain structural complexity.Structural complexity is closely related to biodiversity; complex habitats offer shelter to native species, protect from currents, and facilitate the presence of nutrients in the water.
Several works have focused on the extraction of 3D metrics able to measure this environmental feature at different scales.In (Friedman et al., 2010), Friedman defined the multi-scale metrics of rugosity, slope and aspect on stereo image reconstructed (about 50×70 meters) areas of the seabed using geo-referenced image pairs collected by AUVs.This work also discussed the Figure 1.An outline of the proposed method.Starting from a multi-view stereo point cloud of a reef, we select the point cloud area representing a colony.This area is then used to create an approximation plane.Using the plane position and orientation, we select the most representative photo from the set of calibrated/oriented cameras.By projecting the selected photo onto the plane, we obtain a measurable, quasi-orthorectified image.possibility to use these metrics as descriptors for the automatic classification of marine habitats.In (Young GC, 2017), authors use non-calibrated cameras to investigate middle range areas (2×2 meters) of the reef defining an higher number of multi-scale 3D measures (such us point-to-point distance, linear roughness, fractal dimension and vector dispersion).These quantities extracted from reef's meshes are then compared with in-situ measurements and with known values of ground truth 3D printed objects, placed in a submarine environment and reconstructed with the same procedure.In this way they demonstrated the effectiveness of 3D reconstruction for quantifying structural complexity.Burns (Burns et al., 2016) applied these structural features to high resolution reconstructions from surveys of sample areas before (2014) and after some (2015) disturbance events, like meteorological or bleaching phenomena, cause of mortality among corals.His work aimed at determining with high precision the amount of morphological degradation of the surveyed area in order to evaluate its impact on the diversity and the abundance of the present organism.Regarding the measurement of the growth rate, Ferrari (Ferrari et al., 2017) studied the seasonal growth of hard corals colonies, Bennecke (Bennecke et al., 2016) measured directly from 3D models, generated using 1500 high resolution images collected across 10 m 2 , the growth along 8 years of deep seas corals.
All of those studies focused on taking measurements on species with solid and coherent surface, where photogrammetric analysis can highlight even small changes in size on singles organisms.Different solutions must be adopted if measurements are taken from non-rigid or moving organisms, as in the case of length estimation of fishes in aquaculture stocks.Shortis (Shortis et al., 2009) provided a detailed overview of how non-rigid bodies can be easily measured by the use of stereo cameras.In (Drap et al., 2017) authors developed some effective photogrammetric tools for the monitoring of Mediterranean red corals using a specific designed hardware/software.A 3D printed frame equipped by coded targets provides a local reference system for a series of observation; camera poses were computed using targets and a dedicated interface was developed to take measurements directly from calibrated images.
In our work, we describe and assess a method for taking accurate measurements on P. clavata, a non-rigid, tree-like octocoral.The species shows a flexible skeleton and thin ramifications, which greatly affect the quality of the photogrammetric reconstruction, resulting in a hazy and incoherent cloud, full of "false" points with random orientation.Therefore, the standard procedure to take measurements over a reconstructed, textured surface in 3D space is impractical.Our method allows to overcome this by using quasi-orthorectified images, that are produced by projecting the photos used during the photogrammetric reconstruction on the best fit plane estimated from the point cloud of the colony.While this process does not produce fully metric restitutions, the resulting images can be measured with an accuracy that falls below the requirements for this type of ecological observations.The paper aims to assess the accuracy ad precision in taking dimensional measurements on P. clavata colonies with 2D projected images.This is achieved by testing: 1. the intra-dataset precision, analyzing multiple projected images over the same set of colonies; 2. the inter-datasets precision, analyzing images of the same set of colonies from time series; 3. the accuracy of the measurements, comparing the results with ground truth data collected with methods commonly applied in marine monitoring programs.
Compared to previous works, the presented method does not require a detailed reconstruction of individual colonies but only a comprehensive photographic coverage of the area of interest, enough for a global multi-view stereo reconstruction of this area.
For this task, a low-cost pre-calibrated camera is sufficient.This approach drastically reduces the time spent working on the field.
Considering the importance of the red gorgonian in the ecosystem, as an engineer species, this research will support practitioners and scientists in improving accuracy in monitoring plans.These actions will be complemented by developing a user friendly software for promoting the method as a standard approach to collect relevant metrics over the red gorgonian colonies.

METHOD
The input of the proposed method is a multi-view stereo reconstruction of the underwater habitat (dense point cloud + calibrated and oriented images) and a selection of the point cloud area describing the colony that needs to be measured.
The selection of the colony area from the global cloud may come from different sources: it may be a completely manual selection, an "assisted" selection based on geometrical properties, or the outcome of a completely automated classification/filtering process, able to isolate each colony in the environment.This makes the method versatile and flexible, as it may be used when working on a specific colony, but also as a batch tool (which is a current topic of investigation) for a carpet analysis.
Given the starting data, two main steps are fulfilled: • Starting from the point cloud of the selected colony, an approximating plane that best fits the points is calculated.The fitting plane is calculated using the classic PCA approach: the two directions which spanned the plane are the two largest eigenvectors of the covariance matrix associated to the selected points.
The plane generated is oriented trying to best fit the selected area (thus, it follows the PCA analysis results); on this plane we generate a rectangular region, slightly larger than the colony size, to accommodate for possible photographic detail not covered by the point cloud.This polygon is composed by just two triangles, and it is automatically parametrized.
• Then, one or more images are selected among those which generated the point cloud, and projected on the fitting plane, generating the quasi-orthorectified images.Ideally the best candidates should minimize the angles between the plane normal and the cameras viewing directions: this is our initial filter.Among these cameras which have a nearly frontal shoot with respect to the fan, we then select the ones nearer to the colony.Given the 2-sided nature of the colony, it may be possible to have two "best shot", one from the front, and one for the back.The images in which the subject is partially occluded are manually discarded.
The output of this process is a series of oriented and measurable images, depicting the colony.The reprojected images are now measurable since they are projected on a plane immersed in 3D space, so we can compute quite easily the distance between any pair of pixels of the original images.
At the moment these two steps are carried out using a combination of MeshLab filters (Ranzuglia et al., 2013).While approximating the colony with a single plane seems a harsh simplification, the natural shape of these organisms is plane-like.Additionally, a measurable 2D representation is easier to access and manipulate also by non-3D experts.The final output is clearly not a perfect orthorectified image, but the next section will show that the measurements made on it are coherent and suitable for ecological monitoring.Moreover, the accuracy error may be estimated.

RESULTS
We tested our method on two dense point clouds obtained by an image-based reconstruction of the same underwater site.The two photographic datasets were acquired during two surveys respectively in January (see Figure 2) and October 2017 using a Gopro Hero 4 Black Edition.Each photographic dataset consisted of about 300 photos and covered an area of approximately 50 m 2 characterized by a population of about 100 colonies of the red gorgonians.The reconstructed point clouds (obtained with Agisoft Photoscan) were scaled and aligned in the same reference system using MeshLab (Cignoni et al., 2008).Three sample colonies (specA, specB and specC) were selected from both reconstructions in order to have for each gorgonian a reconstructed point cloud related to the month of January and one to the month of October.Each sample was partially or completely portrayed during each dive campaign by 5-10 shoots (although in some cases the framing is very angled or the sample appears quite far from the camera).
For each of the 6 reconstructed gorgonians we also have a high resolution image with spatial reference (Figure 2).The reference images, although affected by an uncertainty of 2-3 mm due to the tilt and deformation of the reference panel, can be a reliable ground truth for the measurements on the results of the proposed method.
SpecA had the most complex shape between the three considered colonies; as evidenced in Table 4, it was also the most distant to be approximated with a plane.In order to test the intra-dataset precision of the method, two different quasi-orthophoto of SpecA were created by re-projecting the more convenient images from the same dive on the associated plane of best-fit.A series of measures were then collected on the well distinguishable ramifications directly on the two quasi-orthophotos.This procedure, visually illustrated in the first row of the Figure 3, is better analyzed by the Table 1.
The eleven measurements (all integers expressed in millimeters) between two quasi-orthophoto projected from the month of January dataset differ of only a few millimeters.The mean of the relative percentage error calculated using the maximum dispersion between the two measures settles around 1.5%.In the worst case, on the M 5 measurement, the relative percentage error is 3.9% which corresponds to a precision of less than 3mm; on the contrary in the best case, in M 0 and M 9, which both have a relative error of 0.4%.the precision is lower then a millimeter (in our case, the millimeter is the significant digit).
While, as seen, the Table 1 test the first assessment the Intradataset precision, Table 2 assesses the third requirement upon the accuracy of the method.The same eleven measures were taken on the branches of the same colony from the month of January reference image (see second row of Figure 3) and then compared with the mean values of the related measurements in the quasiorthophotos.The mean value of the accuracy stands around the 3.3%, which corresponds to a little less than 5mm for M 0. This accuracy estimation, although a little rough because obtained using only a couple of measurements to compute the average of January values, in these types of surveys is already considered acceptable.More in general, measurement errors below one centimeter are considered acceptable.It should also be noted that the difference in terms of resolution between the ground truth image and the images acquired during the survey is very high.
The last requirement, the Inter-dataset precision, is proved by the ison between the quasi-orthophotos generated for the same fan (SpecA) by the two temporal datasets.In Table 3 a dataset of new eleven measurements were collected between the two generated quasi-orthophotos for the month of January and for the month of October.The set of measurements has been changed specifically to not include measures taken on the branches tips which may have grown up in ten months.In this case the mean relative error is around the 0.7%.
This procedure for the evaluation of the error has also been applied to other colonies, SpecB and SpecC.
Figure 5 shows that coherent measurements can be obtained also when the quasi-orthophoto is obtained from small portions of the original image.Nevertheless, Figure 6 shows that when the resolution of the projected images is too low, it may be difficult to obtain the measurements: in this case, the measurement error w.r.t. the reference may be above the centimeter.
The quality of the quasi-orthophoto may be related not only to the resolution and the blurriness of the projected portion of the image, but also to at least two other aspects.First, perspective deformation errors might affect the projected image on the plane of best fit; this error increases as much as the camera is angled with respect to the plane.Nevertheless, it is possible to know in advance the quality of the projection of the image.Second, there is the fitting error of the plane respect to each sample of the point cloud.What most influences the choice of the approximate plane is not the noise of the point cloud, its density or the partiality of the reconstruction, but shape of the gorgonian.The calculation of the approximate plane with RANSAC, generally a more robust to noise method, does not highlight appreciable differences in terms of RMS (see Table 4) for the closest to being planar (SpecB and SpecC) considered colonies.
To a small variation of the inclination of the approximate plane corresponds a very small variation of the view-angle between the plane and the chosen camera, therefore, the choice of the plane does not significantly influence the perspective projection error.
In fact, in the worst case in which the gorgonian has the most complex shape specA, and the RMS of approximation is the highest, we have that the dihedral angle between the plane calculated with the coovariance matrix and the chosen camera is 44.86 degrees, while between plane calculated using the RANSAC method and the same camera is 44.11 degrees.
The more the gorgonias tend to be three-dimensional, the more the approximation error increases and the choice of the approximation method tend to give dissimilar results in term of RMS.However, the best fit method seems to take better account of changes in the shape of the colonies.Since the central region of the gorgonians is usually more planar, another solution could be to create a plane with a greedy approximation method starting from the center.This option, built to privilege some regions respect to the others, generates planes with RMS even higher than previous ones the cases of less planar colonies.
The fitting error can be locally estimated (as Figure7 shows) in every part of the projected image.Hence, projection and fitting qualities may help in defining the "reliability" of measurements, and guide the calculation of the global statistics over an entire area.

DISCUSSION AND CONCLUSIONS
P. clavata presents all the characteristics to be considered an effective indicator of the impact of climatic anomalies on the coralligenous community (Linares et al., 2008) which represents one of the greater biodiversity hotspot habitat in the Mediterranean Sea (Ballesteros, 2006).Therefore, demography studies over gorgonians populations are needed fro providing relevant information on the conservation status the entire habitat.
The key limitations for estimating gorgonians morphometries like maximal height, branches lenght, epibiosis percertage, include the depth ranges of distribution for the species (from -25 m to -150 m) and the spatial extension of the populations.In those conditions, the traditional sampling strategies based on scuba diving The semi-planarity of the colonies allows us to switch from a wide area survey of the seabed to a local analysis.Measurements taken from the quasi-orthorectified projection are comparable to those extracted from the reference images that portray each colony individually.The proposed method is based on rapid photogrammetric surveys for sampling data across wide areas (50 m 2 ) during one single dive operative time.A large-scale acquisition, much faster then a colony-by-colony detail survey, is well suited to the need of frequently monitoring the health status of organisms sensitive to seasonal variations in temperature.Furthermore, the proposed method makes it possible to evaluate the temporal change of the colonies otherwise not detectable directly The idea of using the almost planar shape of the gorgonians to approximate their surface allows us to extract, in an efficient way, measurements also in the vicinity of the specimen boundary, generally coinciding with the growth regions and with the most incomplete and noisy parts of the reconstructed point clouds.
The estimated accuracy is smaller than the annual maximum height growth rate which has been estimated ranging from 1.8 cm to 2.7 cm yr -1 (Mistri and Ceccherelli, 1994).Finally, our output is a measurable 2D object, which combines the advantages of 2D (higher resolution data, possibility to apply common image processing techniques to extract knowledge), while maintaining other information such as scale and orientation coming from a three-dimensional environment.
Figure 7.The quasi-orthophoto of specC in the October dataset and the the overlay of the best-fit approximation error mapped in each points of the cloud.Pink points correspond to a variation of about +14mm respect to the white ones, in which the interpolation is considered good, yellow ones to a a variation of −14mm .However, the perspective projection of a difference of two centimeters on the best-fit plane results in negligible errors.

IMPROVEMENTS AND FUTURE WORK
While the first results of the method look very promising, several improvements and future developments can be sketched: • This current metric for image selection and projection is generic, and has been designed for standard texture mapping.Improvements taking into account occlusion and quality of images may lead to higher quality results.
• The reprojected image could be a promising input for the automated tracing of the branches, thus generating a vectorial representation of the structure of the colony.
• Ad-hoc solutions for plane estimation in the case of peculiarly shaped gorgonians may help in being able to take accurate measurements also in difficult cases.
All the sea fan examined in this study were segmented individually by the entire reconstruction of the seabed in a previous moment.However, in some cases it may happen that some colony ramifications can be hardly distinguishable from those immediately adjacent belonging to others fans, causing uncertainty in collecting measurements.In order to overcome this problem a depth map of the scene viewed from the selected camera can be projected onto the approximating plane excluding in this way all the others colonies from the projection.More in general, the re-projection approach may be applied to other types of objects with quasi-planar shape that generate noisy point clouds, like vineyards.

ACKNOWLEDGEMENTS
This work was supported by the PhD scholarship EUREKA; European Social Funds (FSE), Programma Operativo Regionale (POR) 2014/2020 Regione Marche (Italy).The authors would like to thank Dr. Ilaria dalle Mura and Dr. Fabio Benelli for supporting the scuba diving sampling activities and UBICA srl for providing the sampling equipment.

Figure 2 .
Figure 2. Top Left: the reconstructed point cloud of the seabed from the photographic dataset of January 2017 with a selection of specA gorgonian sample (highlighted in yellow).b) Top Right: the specA sample reconstructed from the survey of January (pink color) and of October 2017 (green).Measuring the ramifications of the gorgonian is a difficult task due to the noise of the reconstructed clouds.The detection of possible changes between the two reconstruction could not be derived from the clouds due to the incompleteness of the data.c) Bottom Left: the sample specA in an image of the dataset.d) Bottom Right: the sample specA in the high resolution reference image.

Figure 3 .
Figure3.SpecA Quasi-orthophotos projected from different images of the January 2017 survey (top-row).This two images prove the precision of the measure respect different camera view point (intra dataset).A measure extracted from specA reference image (bottom-row).The length of the measured segment is coherent with the one measured on the quasi-orthorectified images.

Figure 4 .
Figure 4.The temporal consistency of the measure between the two datasets (inter dataset precision).Left: the alignment of the temporal reconstruction of SpecA as descripted in Figure 2 an the related aligned quasi-orthophotos.Center: A close up of the aligned outputs shows the coherence of the measure.

Figure 5 .
Figure 5. Top Left: the dense reconstructed cloud of the sample specB from the dataset of January 2017.Top Right: as the gorgonian specB appears inside one of the best selected images.Middle row: measurements taken on two generated quasiorthophotos (highlighting the intra dataset precision) and on the reference image (bottom row).

Figure 6 .
Figure6.Top Left: the dense reconstructed cloud of the sample specC from the dataset of October 2017.Top Right: as specC appears inside one of the best selected images.The middle row highlights the intra dataset precision between the two October's quasi-orthophotos.However, as shown on the bottom image, it is impossible to establish the coherence respect to the ground truth because of the huge difference in resolution between the output and the reference image.

Table 1 .
Table 3 and illustrated by Figure 4 which shows the compar-SpecA Intra Dataset Precision