MONITORING CORAL GROWTH-THE DICHOTOMY BETWEEN UNDERWATER PHOTOGRAMMETRY AND GEODETIC CONTROL NETWORK

Creating 3-dimensional (3D) models of underwater scenes has become a common approach for monitoring coral reef changes and its structural complexity. Also in underwater archeology, 3D models are often created using underwater optical imagery. In this paper, we focus on the aspect of detecting small changes in the coral reef using a multi-temporal photogrammetric modelling approach, which requires a high quality control network. We show that the quality of a good geodetic network limits the direct change detection, i.e., without any further registration process. As the photogrammetric accuracy is expected to exceed the geodetic network accuracy by at least one order of magnitude, we suggest to do a fine registration based on a number of signalized points. This work is part of the Moorea Island Digital Ecosystem Avatar (IDEA) project that has been initiated in 2013 by a group of international researchers (https://mooreaidea.ethz.ch/).


INTRODUCTION
The use of underwater photogrammetry for coral modeling has lately become a quite popular issue among marine biologists (Figueira et al., 2015, Ferrari et al., 2016, Storlazzi et al., 2016. In coral modeling we can distinguish several stages of increasing complexity: (a) just taking nice single pictures for visualization, (b) taking images of a single coral or a larger field by using photogrammetric principles and generating a 3D model for visualization, (c) doing the same but for inspection of coral type and shape, and (d) taking image blocks of larger extension for the measurement of coral change over time. Task (d) is by far the most demanding one, requiring a substantial amount of expertise in photogrammetry and geodesy.
Our study site is at the Moorea Island (132 km^2), French Polynesia, South Pacific. Typical coral growth rates here are in the order of 10 -14 mm/year (Bessat and Buigues, 2001). To detect annual to semiannual changes, each model representing one epoch needs to be precise, reliable, and of high resolution. Whenever high accuracy is required, i.e., for an early detection of changing coral growth, absolute accuracy estimates are of importance. We apply a photogrammetric image processing pipeline for each set of images acquired at a respective epoch and combine it with an underwater geodetic network. Control points are needed to be able to separate the datum problem from global changes in the coral field. They are also needed in deformation analysis to provide an undisturbed, fixed frame. While the minimum of seven control point coordinates would be sufficient to cover the seven parameters of a spatial similarity transformation (3D Helmert transformation), we will apply more to be able to also determine some of the parameters of selfcalibration with sufficient accuracy and stochastic independence. Typically, the quality of the photogrammetric network is quantified by introducing underwater Ground Control Points (GCPs) in a local coordinate frame. With an average ground sampling distance in the order of 0.5 to 1 mm/pixel, however, the theoretical accuracy of the photogrammetric network clearly outperforms the geodetic accuracy. As the GCP coordinates define the accuracy of the point cloud geo-referencing, a direct comparison of the different point clouds on the level of their intrinsic accuracy is not possible. Only the combination of geodetic network accuracy with the photogrammetric block accuracy is meaningful.
What looks like a simple task at first instance, to establish a highly accurate underwater control network (point coordinate accuracy of a few millimeter), turns out to be quite difficult. The coral areas (reef and lagoon) are very dynamic environments, were things are moving all the time (water, dispersed and nondispersed material, fish and other creatures, humans with and without motion devices). Also, high accuracy surveying instruments are not made for underwater use. And last but not least, the human is not build to work underwater in a reliable and precise mode.

Geodetic Network
Acquiring a high accuracy underwater network is a non-trivial task and still a topic of current research (Skarlatos et al., 2017), especially when relatively large areas need to be covered. The test field in our case has a size of roughly 16m x 8m with a maximum height difference of 3.8m, illustrated in Figure 1. For repeated measurements, the GCPs were given fixed positions so that the photogrammetric markers can be re-installed on another survey mission. A total of 11 GCPs were installed in the test field area, whereas two control points were damaged between the image acquisition epochs so that only nine common control points could be used. Each GCP was connected to the seabed surface by a clamping bolt that was drilled into a massive rock or a dead coral within the test field. Due to the nature of the environment, GCPs could not be placed directly on top of the clamping bolt because the line-of-sight for measuring distances between neighboring GCPs would no longer be maintained. The GCPs were therefore placed on top of poles.
Typically, an underwater network is established by the principles of trilateration and eventually height measurements using diving computers (e.g., Skarlatos et al., 2017). Here, we also measure point-to-point distances and relative height differences. Distances were measured using a measurement tape as well as a rigid graduated bar. Height differences were acquired by an underwater levelling procedure, using a tripod and an underwater laser (green light, Moray Laser) that pointed to the respective levelling rod, placed on top of the GCP clamping bolt (Capra et al., 2017). Figure 2 shows various devices we used. The length of the poles, separating the photogrammetric targets from the seabed topography was initially set to 76 cm but afterwards lowered to 46 cm for all GCPs. Distances were measured for each pole length configuration while height differences were acquired relative to the clamping bolt, and thus are equal for both, the long and the short pole configuration. Figure 3 illustrates the geodetic network configuration along with the conducted measurements for both configurations and table 1 summarizes the number of respective measurements. Although the geodetic network was measured in both field missions (January and August 2017), only the solution from August was found to be reasonable and therefore used for both epochs.
The local geodetic network was solved using Trinet+ software (Guillaume et al., 2008). Point 14 (closest to the barycenter of the test field) was fixed by its initial coordinates, whereas the orientation of the coordinate frame was defined by constraining the direction between point 14 and point 2 as well as by the definition of the vertical, provided by the measured height differences. The scale is intrinsically defined by the distance measurements. Thus all seven parameters of a 3D Helmert transformation were determined a priori. A popular alternative is to use the principle of free network adjustment. Here, minimization results in a cofactor matrix Qxx that has a minimal trace compared to all others with minimum datum. It therefore delivers optimal results in terms of inner coordinate accuracy (implicit minimal constraints). The problem is, however, that the solution is biased and the solution vector depends on the initial coordinates, i.e., the final datum is drawn from the approximate coordinates of the network points, computed to an arbitrary     (Schmitt 1982). Therefore we prefer to use the free network adjustment only in cases where the structure of a photogrammetric network has to be analyzed using its object space coordinate covariance matrix (Grün, 1976b). Consequently, our choice follows the network adjustment by providing enough information to define the datum. A comparison between the fixed datum solution and a free network solution is nevertheless provided in the result section for reference.
Trinet+ also allows to estimate an additive constant that is added to each distance observation. It acts like a self-calibration parameter if, for example, a systematic error in the length measurements occurred. We include it in the network adjustment.

Photogrammetry
Image acquisition was performed by an experienced diver pointing the camera in nadir direction at an average height of 1.7m above the surface. A Lumix DMC-GH4 camera with an average camera constant of 13.5mm, estimated from the underwater self-calibrating bundle adjustment, was employed in three data acquisitions, i.e. one January 2017 and two in August 2017. In August, image acquisition was repeated twice: once in the long pole configuration, i.e., adopting the long poles for the geodetic network, and once in the short pole configuration. The ground sampling distance was in the order of 0.5 to 1 mm/pixel for both campaigns (see example image in Figure 4).
Photogrammetric processing was carried out with Agisoft PhotoScan (2017) and was performed as a self-calibrating bundle adjustment. The results were further filtered and iteratively optimized (interior and exterior camera parameters) by removing weak image observations, i.e., those with large re-projection errors and those representing object points seen in only two views. By introducing the GCPs with their standard deviations as obtained from geodetic network adjustment, geo-referencing was performed in a softly constrained self-calibrating bundleadjustment. As for the geodetic network adjustment, it is well known that such a fixed datum adjustment in the photogrammetric block does not yield the minimum possible mean object point variance (e.g. Fraser, 1982, Fraser, 1984. Like in the geodetic network, however, we are interested in detecting changes relative to a fixed datum. After the bundle adjustment, the dense image matching step was performed for the two campaigns, i.e. January and August with long poles, obtaining two dense point clouds characterized by a mean resolution of about 2mm. The comparison between the two point clouds is performed in CloudCompare (2017). Differences are computed using the M3C2 plugin.

Geodetic Network
We computed the final solution of the geodetic network iteratively starting with reasonable estimates of the expected measurement accuracies. After each run, the residuals were analyzed, obvious gross errors eliminated, and, if needed, the sigma a priori values slightly adjusted. Table 2 shows the a priori and a posteriori 0 values for the respective measurements, obtained in the last run. The corresponding observation residuals are presented in Figure 5.
Although the height observations for the long and short pole configuration were identical, small differences in the respective residuals are observed, as there is also a contribution of distance observations to the height variations, since we are dealing here with a 3D network. Table 3 presents the results obtained for the geodetic network adjustment, for the long and short pole configuration, respectively. Given are the mean standard deviations, 1 level, the length of the major error ellipsoid semi-axis, and the exterior reliability. The latter indicates the effects of possible undetected gross errors.  The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020", 4-7 June 2018, Riva del Garda, Italy A significant additive constant was detected, i.e., 42.3 mm ± 7.84 mm for the long pole and 20.6 mm ± 3.99 mm for the short pole configuration. The positive sign in both cases indicates that the measured distance were too short in general.
A graphical illustration of the 'fixed datum' solutions is provided in Figure 6. For visualization, the dimension of the error ellipsoids were increased with respect to the network dimension. Also shown are the arrows of the external reliability to highlight the (directional) sensitivity on possible undetected gross errors for each GCP position. As mentioned before, point 14 was fixed (no error) and the direction to point 2 was constrained. The error ellipsoid of point 2 therefore reduces to an error ellipse. Table 3 and the error ellipsoids clearly show the better performance of the short pole network, because the longer the poles the greater the danger of making measurements errors through the touching and bending of poles.

Photogrammetrically derived Point Clouds
The photogrammetric block was solved using Brown's standard model for self-calibration, yielding calibration residuals clearly below the pixel level. The fixed datum solution of the geodetic network yielded RMS errors in the object space of 50.0 mm and 43.5 mm for the January and August dataset, respectively. These estimates are based on eight GCP points: point number 7 was accidentally damaged during the measurements, point 10 was not installed in January, and point 16 was excluded due to an obvious large difference (more than 80 mm) in both datasets. Excluding point 16 is also justified from the geodetic network perspective as it shows a large elongated error ellipsoid (MA = 65 mm) and external reliability vector (ER = 147 mm). In addition, the main component of its large RMSE aligns well with the orientation of the error ellipsoid. Table 4 summarizes the mean RMSE of the 3D components for both, the fixed datum and the free network configuration. A comparison for the short pole configuration is available only for the August dataset. These values are in relatively good accordance with the a priori standard deviations of the GCPs (see Table 3). Estimation theory tells us that under the null-hypothesis (dealing only with random errors) the expectations of the RMSEs should be unbiased estimators of the a priori sigmas (Grün, 1976a, Gruen, 1982.

Change Detection
The comparisons between the geo-referenced point clouds from January and August are presented visually, first in two 2D profiles, and second in a colorized 3D distance map, projected onto the August point cloud. The position of the two profiles with respect to our test area is shown in Figure 7.
Closer inspection revealed a misalignment between the point clouds, therefore an iterative closest point (ICP) registration method, implemented in CloudCompare (2017), was applied to remove any residual misalignment. The resulting profiles, before and after the ICP application, are shown in Figure 8. The transformation matrix computed by the ICP yielded a small rotation and the following translation components (with respect to the August point cloud):  The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020", 4-7 June 2018, Riva del Garda, Italy = 2.1 = 0.2 (1) = 1.2 Figure 9 shows the signed distances between the corresponding points computed by the CloudCompare M3C2 plugin, which implements the Multiscale Model to Model Cloud Comparison method (Lague et al., 2013). The differences are reported on the August point cloud. Blue colorization indicates material ablation while red areas highlight accumulation. Some corals broke off or fell apart, producing topographic changes of 0.5m and more. Close inspection also revealed various areas where the distribution of seafloor material (mostly sand) changed considerably. These two effects are the prominent differences seen in Figure 9, rather than changes in the living corals that are expected to be at least one order of magnitude smaller.
The ICP process computes an average, although robust, transformation that minimizes the distances between all corresponding points in the two epochs. Although the suspected systematic effect obviously becomes smaller (Figure 8), we do not have a better ground truth than the GCP network. Consequently, real changes might be used for minimization such that variations and changes we aim to measure, especially on corals, might not be visible anymore.

DISCUSSION
In an environment where every component is prone to changes (seafloor, corals, rocks, etc.), a direct and valuable comparison between two point clouds of different epochs can only be established by a controlled network of rigid non-moving points.
Using the principles of trilateration and height difference measurements, we aimed for a high accuracy geodetic network that allows an accurate geo-referencing of the point clouds such  Figure 7) with extracted points for the January (blue) and August (red) point clouds. The thicknesses of the profiles are 1 cm each. Profile 1 is along the x-axis before (A1) and after (B1) the ICP registration. Profile 2 is parallel to the y-axis before (A2) and after (B2) the ICP registration. Note the obvious systematic shift that has vanished after the ICP registration. Fig.9: Colorized difference map after the ICP registration. In areas of obvious material ablation or accumulation, differences exceed 0.5m (see text for explanation). The distribution of the differences are indicated in the histogram on top of the color bar. The mean of the difference, its standard deviation, and mean absolute deviation (mad) are ̅ = 0.002 , = 0.069 , = 0.035 .
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020", 4-7 June 2018, Riva del Garda, Italy that a direct comparison is possible. As demonstrated above, such a direct comparison, to the accuracy level of the photogrammetric point clouds, isup to this pointnot possible yet. On one hand, we need higher accuracy in the geodetic network parameters, and on the other hand, we need a reliable method to correct for residual offsets. The following sub-section address the different topics.

Geodetic Network
As demonstrated in the previous section 3 and Table 4, the geodetic network in the short pole configuration produces significantly better results, both in terms of the network itself, but also in the RMS error estimates of the bundle adjustment. On one hand, this is a result of reduced measurement errors (long poles tend to bend more easily). On the other hand, the geometry of the available observations (distance measurements) is significantly better. An obvious example is point 16 that shows a large uncertainty towards point 6 in the long pole configuration ( Figure  6A). This is not the case for the short pole configuration as there are distance measurements between point 16 and point 14 (roughly in the direction of point 6, Figure 3B) that help to significantly reduce the uncertainty in this component. The improved distance measurement configuration also effects the network weakness: While for the long pole configuration the major error ellipsoid axis and external reliability vectors were primarily orientated along the X,Y axis, the orientation of the latter quantities changed towards the Z axis in the short pole configuration ( Figure 6B). This shows that the height differences need to be improved in a next step.
For the given short pole configuration, we run simulations to emphasize the different aspects contributing to a good geodetic network solution. As expected, an improvement of the network accuracy is obtained when the principle measurement precision increases. The effect on the estimated coordinates is summarized in Table 5: it lists the average coordinate a posteriori ̂ level, the average major semi-axis of the error ellipsoids, and the mean external reliability for different simulated a priori observation errors in the short pole configuration. The given values were computed with Trinet+ in the fixed datum configuration. As before the free network solution produces results with a posteriori sigma values that are approximately 25% better. Simulation results also show that the external reliability vectors always dominate the height component as long as the height observation accuracy is lower than the accuracy of the distance observations.

Photogrammetric Accuracy
The general theoretical accuracy of the underwater photogrammetric network can be estimated by the following equation: Being, q the design factor indicating the accuracy of the imaging configuration, m the mean image scale and sxy the mean image measuring accuracy (Luhmann et al., 2013).
For this case study, the mean image scale is about 130, while from the photogrammetric bundle adjustment a value of about 0.6 pixel for the image coordinate accuracy is obtained. Considering a q value corresponding to the weakest camera configuration, i.e. q = 3 (Luhmann et al., 2013), the theoretical accuracy of the underwater network equals 0.8 mm. A further degradation by approximately a factor two can be considered for underwater photogrammetry (Maas, 2015), leading to a final theoretical accuracy of about 1.6 mm, which is still an order of magnitude better than the accuracy provided by the underwater geodetic network (short poles). Also this theoretical value correlates well with a past empirical analysis performed with the same camera system (Guo et al., 2016).
However, it is well known that weak camera network configurations in combination with very large systematic errors of underwater camera systems (e.g. the radial distortion may easily go up to 500 pixels) can entail unfortunate systematic error propagation, leading to strong deformations of the photogrammetric model . The accuracy and robustness of the photogrammetric model can be improved by optimizing the image acquisition strategy, i.e., nadir and oblique acquisition angles combined with a double grid (cross-wise) flight path. Scale bars can be placed on the sea floor to validate the model locally.
Also, more attention should be given to the generation of high quality images. There are many external factors of influence that may results in bad image quality: motion blur, moving reflections of sunlight on the water surface, dark clouds moving in, dispersed material reducing water transparency and image contrast, floating material acting like an uncontrolled ambient light filter and causing back scattering if external lighting sources (e.g., strobes) are used, color refraction of camera system, especially when a flat port is used, etc.
For all the reasons of possible systematic errors, a control network, capable of highlighting error accumulation and nonlinear deformations over the photogrammetric model, is strongly needed. Additionally, when the physical extend of the photogrammetric model grows, the importance of an accurate geodetic network becomes even more important. In this study, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020", 4-7 June 2018, Riva del Garda, Italy the accuracy of the GCP coordinates clearly limit the empirical estimate of the photogrammetric absolute accuracy.

Change Detection
In scenarios where changes at the level of centimeters over timescales of years are to be estimated, the stability and reproducibility of control points should be at least as important as the estimation of their accurate coordinates. Due to the presumed high accuracy of the photogrammetric network in general (here we were rather pessimistic), a fine alignment between the multi-temporal point clouds based on a number of stable signalized extra points might be necessary to detect millimeter level changes in the coral reef.
In the presented work, such a fine alignment based solely on the GCPs was not feasible because different long poles were used in the January and August dataset. While this difference was taken into account for the geo-referencing, the long pole GCPs were found to be not stable enough for the fine alignment.
In the current work, significant height changes (1 level) can be approximated by: where ℎ1 and ℎ2 represent the standard deviation of the height component in the two point clouds. Note that the influence of the photogrammetric object point reconstruction can be neglected, as it is at least one order of magnitude lower than the geodetic network accuracy. In our case, ∆ℎ is in the order of 3 cm. The respective 3D component, derived by the same principles, reaches 6.6 cm.
The photogrammetric object point accuracy can reach millimeter to sub-millimeter levels, especially in the vicinity of the fixed GCP point (number 14 in our case). There is the danger, however, that small differences in mounting the GCPs between the measurement epochs yield a small shift in object space and as such distort the estimated accuracies.
As seen in Figure 9, major changes in the seafloor topography have been detected from processes other than changes of the living corals. For a detailed analysis about coral growth or decline, a suitable masking procedure is needed to isolate the features of interest.

CONCLUSIONS
This paper presents the current stage of our attempt to detect centimeter level changes in a coral reef test field (16m x 8m) at the Moorea Island by photogrammetric means. This requires an accuracy of photogrammetrically produced point clouds of the different measurement periods in the order of a few millimeters. For change detection, a common datum for the multi-temporal point clouds is of high importance. Due to changes of all natural components in this environment, a local geodetic network with stable control points was constructed. Using a suitable tape, a graduated bar, and a green laser based leveling procedure, distances and height differences were measured and used to solve for the unknown point coordinates. While different configurations were tested, the best solution of the geodetic network reached an average level of 1.6 cm for the major semiaxis of the error ellipsoid. As underwater work is exhausting and time-consuming, we demonstrated that an a priori planning of the geodetic network, both in terms of point positioning and measuring, is of high importance. Improving the accuracy of the principle measurements (distances and height differences) also is of essential importance. This aspect is an ongoing research topic that requires further testing.
We also conclude that the absolute accuracy of the photogrammetric result can currently not be accessed, as control points with accuracies in the millimeter to sub-millimeter range would be needed. Therefore, we also cannot detect systematic errors that eventually are present in the point clouds, even after having applied self-calibration in geo-referencing.
Nevertheless, a direct comparison between the two point clouds based on the accuracy of their geo-referencing is possible to the level of a few centimeters. As a follow-up registration based on the iterative closest point (ICP) algorithm provided a more realistic appearance of the observed differences, we suspect a geo-referencing error in the order of one to two centimeters. To overcome the limitation of the underwater geodetic network, we suggest additional stable signalized points upon which a fine registration, independent of the geodetic network, can be performed.
Based on theoretical considerations regarding the accuracies for photogrammetrically reconstructed object points, the detection of centimeter level changes over test fields with more than 10 m in dimension, is feasible. One of the biggest challenges in these environments, however, is to materialize a good number of well distributed 'stable' reference points, which is the key for successfully detecting coral reef changes.
We can finally summarize the most important aspects for a high precision geodetic underwater network for photogrammetric applications as follows:  Stable GCP points: GCP anchoring on the seafloor must be stable for long periods to allow coral growth monitoring over long time scales. When GCPs are attached, stability must also be provided during the measurement of their coordinates and during the photogrammetric image acquisition.  Accurate measurements: The principle measurements for deriving the geodetic coordinates should naturally be as accurate as possible (in our case in the order of a few millimeter). There is still room for improvement and future developments will show if precision and reliability will be sufficiently good to directly compare two point clouds based on their definition of the datum.  Good geometry: Measurement connections in both distances and height need to be optimized for any given scenario. While the topography in the field of interest might prohibit important connections, its effect should be included at the planning stage. For areas that roughly have a planar geometry by nature, it might be helpful to artificially increase the height of a few GCP points such that the 3D network gains accuracy.  Height component: The vertical axis is generally the more important, but also the more critical component for coral measurements, especially if image acquisition was conducted with a nadir looking camera. This is due to the possible high correlations between orientation parameters and some of the self-calibration parameters. It is therefore of special importance to get precise height estimates.  Minimization of mechanically related effects: we suggest to attach the GCPs as close to the topography as possible. Ideally, GCPs should be mounted vertically on short poles (as short as possible to guarantee neighborhood line-of-sight visibility) for measuring the geodetic network. For the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020", 4-7 June 2018, Riva del Garda, Italy photogrammetric network, however, they should be attached with zero pole length. Using a zero-pole-length configuration during image acquisition also reduces the danger to accidentally damage a GCP and, in post-processing, can be used to do a fine point cloud registration solely based on these targets (rather than a general ICP).  Additional signalized points: In addition to control points of known position, signalized points with unknown coordinates can be placed in various locations of the test area, serving as check points between different acquisitions but also for a fine registration. Finally, only such a fine registration will allow to compensate for the inaccuracies of the geodetic network and, if systematic errors of the photogrammetric block adjustment can be kept at a small level, will allow to reliably detect centimeter to sub-centimeter changes in the coral reef environment over long time periods.