ANALYSIS OF LOW-LIGHT AND NIGHT-TIME STEREO-PAIR IMAGES FOR PHOTOGRAMMETRIC RECONSTRUCTION

Rockfalls and rockslides represent a significant risk to human lives and infrastructures because of the high levels of energy involved in the phenomena. Generally, these events occur in accordance to specific environmental conditions, such as temperature variations between day and night, that can contribute to the triggering of structural instabilities in the rock-wall and the detachment of blocks and debris. The monitoring and the geostructural characterization of the wall are required for reducing the potential hazard and to improve the management of the risk at the bottom of the slopes affected by such phenomena. In this context, close range photogrammetry is largely used for the monitoring of high-mountain terrains and rock walls in mine sites allowing for periodic survey of rockfalls and wall movements. This work focuses on the analysis of low-light and night-time images of a fixed-base stereo pair photogrammetry system. The aim is to study the reliability of the images acquired over the night to produce digital surface models (DSMs) for change detection. The images are captured by a high-sensitivity DLSR camera using various settings accounting for different values of ISO, aperture and time of exposure. For each acquisition, the DSM is compared to a photogrammetric reference model produced by images captured in optimal illumination conditions. Results show that, with high level of ISO and maintaining the same grade of aperture, extending the exposure time improves the quality of the point clouds in terms of completeness and accuracy of the photogrammetric models.


INTRODUCTION
Temperature variations occurring during different environment conditions can contribute to rock instabilities phenomena such as rockslides and rockfalls. Indeed, significant temperature changes between day and night and temperature cycles, can seriously affect the conditions of the materials filling the rock mass features and contribute to the variation of the stress distribution within the rock mass. This can result in rock mass deformations and eventually the detachment of blocks and debris from the rock surface. Rockfalls can be characterised by significant levels of kinetic energy and therefore they represent a serious risk to human lives and infrastructure located at the base of the affected rock surfaces. Close range photogrammetry for the monitoring of the rock surface combined with a thoughtful geostructural characterization, is a necessary step to assess the potential hazard, improve the prediction capability and design appropriate mitigation measures to reduce the risk in the areas potentially affected. Photogrammetry and remote sensing techniques, in particular, are used for the monitoring of particularly unstable or critical rock slopes. These techniques not only allow for the continuous monitoring of the slope but they can also provide information on the three-dimensional (3D) terrain geometry, the 3D terrain displacements and the volume of the moving or detached terrain. The continuous measuring of displacements on the rock surface significantly contributes to the understanding of the dynamic processes and to the early recognition of potential impacts on human and natural systems. In some cases, photogrammetric stereo-plotting (Van Westen et al., 2003) or the comparison of * Corresponding author multi-temporal digital surface models (DSMs) obtained from automatic surface reconstruction techniques (Dall'Asta et al., 2017) allow monitoring unstable slopes, identifying the potentially unstable blocks and estimating their volumes. Nevertheless, the identification of a survey method and a lowcost measurement control system capable to reliably assess the expected displacements and volumes is not always straightforward (Scaioni et al., 2014). In the past, photogrammetry has been successfully applied to monitor the evolution of landslides. Some of the systems include aerial photographs (Casson et al., 2003;Mora et al., 2003) and high-resolution UAV (Unmanned Aerial Vehicles) imagery (Mancini et al., 2013;Dall'Asta et al., 2017) for detecting deformations or changes over large areas. In addition, other systems based on close range terrestrial photogrammetry are mainly a remote alternative to traditional discontinuity measurement techniques of periodic survey of landslides or movements (Sturzenegger and Stead, 2009;Stumpf et al., 2015). In this field, fixed high-resolution optical cameras have been developed for permanent monitoring over small areas of active landslides. In particular, the use of a fixed-base stereophotogrammetry monitoring system was successfully adopted to evaluate the movements of a landslide by Roncella et al. (2014). A similar system is currently being tested with the aim to provide continuous image acquisition for the day and potentially also the night. In fact, the 24h/day monitoring would allow a deeper understanding of rock slope instabilities especially due to temperature changes and other environmental effects.
This work focuses on the analysis of low-light and night-time images of a fixed-base stereo pair photogrammetry system. The study consists of exploring different settings of a DSLR (Digital Single Lens Reflex) camera during the acquisition accounting for different values of ISO, aperture and time of exposure. This allows investigating the effect on the quality and the reliability of the images to produce accurate DSM for change detection. Ultimately, the current study aims to find the best setup for the night-time acquisition of images in a mine site which can be used for accurate and continuous rock slope monitoring.

METHODOLOGY
The characterization of rock slopes is often constrained by accessibility and safety issues. Consequently, terrestrial remotesensing techniques represent promising alternatives and supplements to traditional rock engineering survey techniques. Depending on the purpose of the work, one may wish to survey control points or fixed installed prisms on the rock slope. However, this operation can quickly become the most timeconsuming component of a project. Structure-from-motion (SfM) workflows are based upon algorithms for efficient and automatic orientation of large image sets and more automated surface reconstruction. Nevertheless, the importance of carrying out well-established fieldwork strategies, using proper camera settings, ground control points and ground truth for understanding the different sources of errors, still needs to be adapted in the common scientific practice. These new techniques have a great potential to provide topographic information for geoscience applications at significantly lower costs than classical topographic and laser scanning surveys. The use of a fixed-base stereo-photogrammetry system for monitoring application requires an extensive analysis of theoretical accuracy depending on the network geometry, i.e., the relative positions of the camera and the object, and the available hardware. The quality of the images is of great importance in the stereo-matching for the 3D surface reconstruction, especially for low-light and night-time acquisitions. Indeed, it is well known that photogrammetry works well with good illumination and no shadows but the image acquisition at low light conditions and especially during the night is a big challenge. In such cases, the photogrammetric system needs to be equipped with a highsensitivity DLSR camera. After the system is installed, the camera settings for the best level of exposure must be determined. The study of the influence of parameters, ISO, aperture and shutter speed, can be conducted for this purpose (Verhoeven, 2016). The ISO value expresses the sensitivity of the sensor to the incoming light. The basic value is ISO 100. It is true that high ISO values imply less resolved details and more image noise, i.e., the lower the ISO setting, the less noise will be found in the image and the higher the image quality. The disadvantage is dealing with a less light-sensitive device. On the other hand, high ISO values increase the light-sensitivity but it can also result in noisy and grainy images. The aperture defines the amount of light allowed to pass through the lens. The higher the aperture, the greater is the amount of light passing through the sensor. The device that physically limits the angle of rays passing through the whole camera system is the diaphragm. The aperture is specified as an f-number (ratio of focal length to effective aperture diameter) and each lens has a set of marked "f-stops" that the f-number can be set to. It is important to note that less light passes with larger f-number because the aperture number is inversely related to the light collected. In addition, full f-stops increase or reduce the illumination by a factor of 2 and each step is called a stop or exposure value (EV). The aperture also regulates the amount of incoming light whereas the shutter speed regulates how long the sensor is exposed to the light. The duration of the time of exposure is measured in seconds or fraction of seconds. Aperture and time of exposure work together and usually if one is changed, the other has to change in the opposite direction. For taking into account the EV in the analysis of the results, it was calculated for each model as: where S represents the ISO value of the model, N is the aperture and t is the shutter speed. In our test, a full format DLSR camera Nikon D800 with a resolution of 36 Mpixel and a wide sensitivity ISO range of 100 to 6400 was used. Camera settings were selected to provide the best image quality, including medium to high ISO light sensitivity settings (200, 400, 800, 6400), from low to high f-stop (8, 4, 2) and exposure times from 1/15 to 30 s in low light environment, as reported in The experiment consists of a stereo pair with a fixed base length acquiring images before, during and after sunset. The images were captured with the DLSR camera using various settings, accounting for different values of ISO, aperture and time of exposure. For each acquisition, the DSM was generated in Agisoft PhotoScan version 1.3.0 (Agisoft Photoscan, 2018) and then compared to a photogrammetric reference model that was produced by images captured in optimal illumination conditions. The comparisons of the generated point clouds were performed with CloudCompare version 2.9.1 (CloudCompare, 2018) using the Cloud-to-Cloud Distance (C2C) tool and the Multiscale Model to Model Cloud Comparison (M3C2) plugin (Lague et al., 2013) for measuring the absolute distances and the signed distances respectively.

Site description and data acquisition
A test survey was carried out on a rock wall that belongs to the cliff of Leverogne (AO) at Valle D'Aosta in Italy. The rock wall is located on the southern side of Valle D'Aosta and it is bounded by NW-dipping tectonic mélanges and lies onto blueschist facies ophiolites. The rock wall is about 90 m high and 100 m wide. A total of 30 stereo pairs were acquired during 3 hours and 30 minutes at a distance from the rock wall of approximately 140 m and with a base-length of about 25.9 m. Images were acquired using the full format 36 Mpixel resolution camera (4.9 µm the pixel size) equipped with a fixed 35 mm focal lens (AF-S Nikkor 50mm f/1.8G Lens) in a slightly convergent geometric configuration. Considering a pixel precision of the area-based matching algorithm of ±0.4 pixel, a final depth precision of 4.2 cm is estimated by using the formula of stereocamera photogrammetry in a normal case (Kraus, 2007). The corresponding Ground Sampling Distance (GSD) is about 2.0 cm. In particular, the images were captured before, during and after the sunset from 18:20 to 21:00, using ISO values from 200 to 6400 in ascending order as the daylight decreased. The grade of aperture was fixed at 2, 4 and 8.
Differently from the fixed-base stereo-photogrammetry monitoring system presented by Roncella et al. (2014), where the exterior orientation parameters are well known, in the present work the relative pose is slightly different for each acquisition because the same camera was used for each stereo-pair acquisition, i.e., the camera was moved between the acquisitions. A network of ground control points (GCP) was measured using a Topcon IS203 total station and used for georeferencing of the models. The GCP network consists of 31 natural control points, shown in Figure 1. Natural features are generally not recognised automatically by the processing software and need to be selected manually by the operator at least on a couple of images. Hence, natural features should be selected very carefully and they should be away from sharp edges or holes to avoid the risk of selecting points which are almost the same on the image but have different spatial coordinates (Thoeni et al., 2016). In average, only 5 control points were used for georeferencing of the stereo-pairs, because their manual individuation on the photos was not easy, especially for the images during the night session. Indeed, low light conditions and high level of ISO values generate a different optic effect of the terrain (see Figure 7, were RGB point cloud models are reported). Afterwards, although the object and the operator are still the same, the collimation of the points is not trivial. Furthermore, terrestrial laser scanning (TLS) using a long range Leica C10 was used to survey the rock wall. The TLS data provides a ground truth model, which was compared to the reference photogrammetric model taken at optimal lighting conditions. Four scans were acquired with high resolution (i.e. 0.05 m of ground resolution at 100 m of distance from the object) in the same reference system of the ground control network. The four point clouds were then merged into one point cloud using CloudCompare.

Image orientation
The bundle block adjustment for the reference model acquired in optimal light conditions, using ISO 200, low aperture with f-stop 8 and shutter speed of 1/250 seconds was performed using Agisoft Photoscan in self-calibration mode. 4 GCP, located at the border of the cliff, were collimated on the stereo-pair and used for georeferencing the model. Table 2 summarises the camera settings and main features of the reference model. Initial tests using bundle block adjustments in self-calibration were conducted for all remaining stereo-pairs. However, the results were not reliable and very high residuals on the coordinates of the GCP, especially for images with low light conditions, were observed. Hence, the interior orientation parameters estimated for the reference model were used as precalibrated parameters for the other 29 stereo-pairs. Overall, the geometry configuration and the focal lens were fixed during the whole experiment. Depending on the quality of the images, the automatic extraction of features in the SfM produced tie-point clouds that were more or less dense.

DSM generation
All the dense point clouds were generated at high level of details in PhotoScan, but depending on the quality of the images, their density is changing between 15 and 50% in respect to the density of the reference photogrammetric model shown in Figure 1. Furthermore, a quick qualitative analysis of the point clouds revealed that the noise increased with decreasing point cloud density.

3D model accuracy and reliability evaluation
The accuracy of the reference photogrammetric model was assessed in CloudCompare by comparing it to the TLS model. Note that areas with vegetation were excluded from the analysis. The models were registered using the iterative closest point (ICP) algorithm. Then, the distances between the photogrammetric and TLS model were calculated using the C2C algorithm by taking the photogrammetric model as reference dataset. A simple Delaunay triangulation technique with a least-square local plane fit was used to interpolate the reference model. In order to evaluate the reliability of the other 3D models, all the point clouds were compared to the reference model using the C2C algorithm and the M3C2 plugin implemented in CloudCompare for measuring the absolute distances and the signed distances respectively. Both C2C and M3C2 methods do not require gridding or meshing of the data. The main difference between the two methods is that the C2C method relies on the point density, instead the M3C2 method is robust to changes in point density and point cloud noise but needs calculation of surface normals (Lague et al, 2013). In particular, when the C2C method is used for each point of the second point cloud, a closest point can be defined in the first point cloud. In its simplest version, the surface change is estimated as the distance between the two points, although the surface can also be modelled as the least square best fitting plane. In fact, in the experiment, we use the 2.5D triangulation method that fits a plane through the nearest point and its neighbour using a Delaunay triangulation starting from the original 3D points as vertices to get a 2.5D mesh. The distances were computed considering a maximum distance of 0.5 m. The M3C2 method measures signed distances directly between two point clouds on particular points, called core points, to speed up the computations. Lague et al. (2013) give a comprehensive explanation of the M3C2. Basically, the distances are computed based on the definition of a diameter of a sphere, called normal scale, centred on each core point to compute a local normal, and a projection scale and a max depth, the diameter and the height of a cylinder oriented by the normal respectively, in which the points of the second cloud will be searched for. In the analysis, the signed distances were calculated using a normal scale of 0.5 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020", 4-7 June 2018, Riva del Garda, Italy m, a projection scale of 0.25 m and a max depth of 2 m for each comparison.
To obtain accurate and comparable results, for each computational method, non-overlapping parts of the point clouds and area with vegetation were removed manually. In addition, all the point clouds were registered to the reference model using the ICP to avoid small systematic residual misalignments due to errors in the georeferencing. Indeed, each model was generated from a different set of exterior orientation parameters because the GCP used were not always the same as their selection strongly depended on the quality of the images (i.e., depending on the camera settings the quality of the images changed and, hence, some GCP were visible whereas other were not).

RESULTS
For the comparison between the reference photogrammetric model and the TLS model, the mean value and the Root Mean Squared Error (RMSE) are provided to assess the accuracy of the overall photogrammetric survey. The remaining 29 point clouds were compared to the photogrammetric reference model and the RMSE of the distances for each computational method is reported.
The subsequent sections provide an overview of the results of the accuracy assessment comparing the reference photogrammetric models to the TLS model (Section 3.1), the reliability assessment of the photogrammetric models (Section 3.2) and an analysis of the colour maps (Section 3.2).

Accuracy assessment of the reference photogrammetric model
The distances between the TLS model and the photogrammetric model were calculated in a range of a maximum distance of 0.10 m, considering the estimated precisions of the instruments. The mean value of the absolute distances between the photogrammetric reference model and the TLS model indicates a value of 3.4 cm and a standard deviation of 2.6 cm. The accuracy of the photogrammetric model, i.e., the RMSE, is 4.3 cm and corresponds to the expected precision calculated in Section 2.4. In general, an improvement of the accuracy can be obtained fixing the position of the camera poses. Indeed, this may partially assimilate part of the errors, at least the systematic components due to the georeferencing. The positions of the camera can be used for direct georeferencing of the model instead of the ground control points, which are generally difficult to recognize on the images considering the particular light conditions. In addition, fixing the camera positions in the bundle block adjustment would reduce the uncertainty about the altered position of the GCP due to possible temperature changes.

Reliability of the generated photogrammetric models
The comparisons were carried out using the C2C method and the M3C2 method in CloudCompare for measuring the absolute distances and the signed distances respectively. In general, the two tested methods for the distance computation (C2C and M3C2) provide similar results, although the C2C method generally provides slightly lower error estimates, especially for the night session. However, the M3C2 is a better indicator for potential systematic errors as it provides a signed distance. In addition, it works better if the point cloud densities are very different (Lague, 2013) but it requires more input parameters. The RMSE of the distances of the photogrammetric models are reported in Table 3 according to the time of their acquisition for both the C2C and M3C2 method. The following analysis focuses on two main sessions: the lowlight session (before and during the sunset) and the night session (after the sunset). Note that the sunset was approximately at 19:35.  Table 3 -RMSE of the distance values for the 29 photogrammetric models computed with the C2C and M3C2 method sorted by the acquisition time of the images.
When looking at the comparisons for the models built with ISO 200 and 800 it can be seen that there is hardly any difference. These images were taken between 18:00 and 19:30 in low-light conditions. The results are very similar despite the different settings (Table 3).
In particular, the comparisons show a mean value of the RMSE of the distances of about 5.8 cm for the C2C and of 6.7 cm for the M3C2 method. From a qualitative point of view, all the models before the night session look quite complete and with no evident noise. The two tested methods for the distance computation (C2C and M3C2) provide very similar results, though C2C results in slightly lower error estimates (the maximum RMSE difference between both methods is 0.9 cm). Considering the night session, the 3D models were grouped by the time of the evening twilight of the day of the acquisition. According to the evening civil, nautical and astronomical twilight, the RMSE of the distances computed with the C2C method are shown in Figure 3, Figure 4 and Figure 5 respectively. In general, during the no light sessions, from 19:45 to 21:30, the mean value of the RMSE increased to 10.9 cm for the C2C and to 12.2 cm for the M3C2 method. The C2C method gives again lower error estimates as the M3C2 method. The maximum RMSE difference between both methods is 1.3 cm. This disparity on the estimated accuracies of the methods is likely due to the different algorithms they rely on. Indeed, it seems that generally the C2C applies more smoothing then M3C2, and, hence, gives lower distances. In the following, a more detailed analysis using the RMSE absolute distances calculated with the C2C method is presented. It should be noted that observations using the M3C2 method would be similar. Over the evening civil twilight session, the ISO ranged between 200 and 800, and slight differences can be noted in Figure 3. The mean of the RMSE is approximately 6.5 cm, slightly higher than the previous session with better light conditions. During evening nautical twilight and as indicated in Figure 4, the range of ISO is from medium to high, i.e., 800 and 6400. There are more discrepancies especially when high level of ISO is used, although there is also a peak of the 3D model 4000_2_1/2 acquired at 20:15 with ISO 4000. At the same time the model with same ISO and aperture of 4 f-stop and 8 f-stop return better accuracy. Finally, the acquisitions during evening astronomical twilight show that, generally with high level of ISO and maintaining the same grade of aperture, extending the time of exposure improved the quality of the point clouds, in terms of completeness and accuracy of the photogrammetric models (see Figure 5, Figure 7 and Figure 8). Some models that are noisier and more incomplete in respect to the reference model represent exceptions. As in the ISO 6400 with aperture 8 series, the model generated with time of exposure of 5 s is the less accurate of the sequence. The deterioration of the mean values of the accuracy of the 3D models during the night session is almost twice compared to the value during the low-light session. However, to get the optimal settings to be attributed to the cameras in order to obtain DSMs comparable with the reference model of daylight hour, it is important to take into account also other features such as the EV.
Evaluating the same value of the exposure, as reported in Figure  6 where the EV is 6, the RMSE of the absolute distances can be of the same order of magnitude for the models acquired during low-light conditions and night-time. Indeed, if ISO is set on a medium high value, e.g., 4000, the accuracy of the corresponding model is 6.4 cm, quite similar to that of the model acquired in low-light conditions with low and medium ISO value, 6.5 and 6 cm for 800 and 200 respectively. The same behaviour can be seen in Figure 6 where the absolute distances are computed for the models with an exposure value of 5. For a high ISO value, e.g., 6400, only a long time of exposure can reduce the errors.       Figure 8 shows the colour maps in two colours of the differences between the reference model and the point clouds generated from the night session in a range of ±20 cm computed with the M3C2 plugin. The colour map in Figure 8a represents the more accurate model (ISO 6400, aperture 4 and time of exposure 10 s), while the comparison in Figure 8b (ISO 6400, aperture 2, time of exposure 10 s) indicates a less accurate model. It can clearly been seen that for the latter the model in the central area is always in front of the reference model. On the contrary, along the lateral areas, the model is always behind the reference model. Probably systematic errors, due to the interior orientation, are still present and the low quality of the images has affected the accurate generation of the dense clouds.

CONCLUSION
The study investigates the use of low-light and night-time stereopair photogrammetry used for 3D reconstruction of a rock wall with different settings of the camera during the acquisition. This analysis aimed at identifying the optimal settings to be attributed to the cameras, in terms of sensor sensitivity (ISO), time of exposure and degree of aperture of the shutter, in order to obtain DSMs comparable to the reference model created with images captured during daylight hours. The comparisons were carried out using the C2C method and the M3C2 method in CloudCompare for measuring the absolute distances and the signed distances respectively. In general, the two tested methods for the distance computation provide similar results. In particular, the M3C2 method is useful to identify some residual systematic errors as it calculates signed distances. When looking at the comparisons for the models built during the low light session, it can be seen that there is hardly any difference. Furthermore, from a qualitative point of view, all the models before the night session look quite complete and with no evident noise. For the night session instead, generally with high level of ISO and maintaining the same grade of aperture, extending the time of exposure improved the quality of the point clouds, both in terms of completeness and accuracy of the photogrammetric models. Finally, it can be concluded that the optimal settings to use during the low-light and night-time sessions are medium high ISO values instead of high levels, such as 4000 and over. This can generally improve the accuracy of the 3D models. In addition, long times of exposure are recommended for achieving the best outcomes. Based on the outcomes of this study, a fixed-base stereo-pair photogrammetric system is currently been installed at a mine site in NSW Australia. The system is used for the monitoring of rockfalls and other slope instabilities. Data acquisition is ongoing and more relevant results will be discussed in future publications.