VERSATILE MOBILE AND STATIONARY LOW-COST APPROACHES FOR HYDROLOGICAL MEASUREMENTS

In the last decades, an increase in the number of extreme precipitation events has been observed, which leads to increasing risks for flash floods and landslides. Thereby, conventional gauging stations are indispensable for monitoring and prediction. However, they are expensive in construction, management, and maintenance. Thus, density of observation networks is rather low, leading to insufficient spatio-temporal resolution to capture hydrological extreme events that occur with short response times especially in small-scale catchments. Smaller creeks and rivers require permanent observation, as well, to allow for a better understanding of the underlying processes and to enhance forecasting reliability. Today’s smartphones with inbuilt cameras, positioning sensors and powerful processing units may serve as wide-spread measurement devices for event-based water gauging during floods. With the aid of volunteered geographic information (VGI), the hydrological network of water gauges can be highly densified in its spatial and temporal domain even for currently unobserved catchments. Furthermore, stationary low-cost solutions based on Raspberry Pi imaging systems are versatile for permanent monitoring of hydrological parameters. Both complementary systems, i.e. smartphone and Raspberry Pi camera, share the same methodology to extract water levels automatically, which is explained in the paper in detail. The annotation of 3D reference data by 2D image measurements is addressed depending on camera setup and river section to be monitored. Accuracies for water stage measurements are in range of several millimetres up to few centimetres.


INTRODUCTION
It is impossible to imagine today's weekly news without hearing the term of "climate change".Beside warming effects, a rapid increase of global flood events can be observed [Hirabayashi et al., 2013, Milly et al., 2002].Thus, heavy rainfalls can trigger inundations or landslides, being dangerous for human and enviroment [Müller and Pfister, 2011].Flash floods are often not predictable and may affect smallest catchments in very short times.However, the costs for permanent overall observation would be excessively high, which is why gauging stations are economical installed.As a consequence, hydrological networks may have an insufficient coverage for threatened areas in case of need.
Improvements in this sense can be achieved using photogrammetry for contactless monitoring of environmental parameters (e.g.[Eltner et al., 2018, Eltner et al., 2017, Schwalbe and Maas, 2017,Capolupo et al., 2015]).With the aid of a single camera and methods for image processing, characteristics of a prevalent situation can be captured and further examined.Its implementation will be carried on with image-to-geometry registration.Thus, the analysed 2D image data is transferred into object space to annotate prior acquired 3D point sets or digital surface models (DSMs).For the development of a versatile mobile and stationary low-cost system for hydrological measurements, the image data of a monitored river section, captured by smartphone or Raspberry Pi camera, is analysed for its prevalent water line.Due to the intersection of image and object data, the water line can be interpreted as series of water levels.

✝ Corresponding author
The following chapter (1.1) provides a small overview of related work concerning novel and conventional approaches for water stage measurements.Section 2 introduces smartphones and low-cost Raspberry Pi camera modules as versatile measurement units.Three study regions situated close to conventional water gauges, one for mobile and two for stationary water level observation, are chosen to enable evaluation of the results (section 3).In section 4, the acquisition of 3D reference data as well as 2D imagery for water line investigation is addressed.Section 5 gives an introduction into the methodology.Finally, the results are evaluated and discussed in section 6 and an outlook is given in section 7.

Related work
The determination of water surfaces from remote sensing data is well-established in geosciences, f.e. using (multi-)spectral indices [Lillesand et al., 2014, Govender et al., 2007, Melgani and Bruzzone, 2004].Nevertheless, these approaches suffer from high instrumental efforts and reduced spatial and temporal resolution to observe small catchments or sudden events (e.g.floods).Unmanned aerial vehicles (UAVs) with low-cost equipment are a considerable alternative [Koschitzki et al., 2017, Gonc ¸alves and Henriques, 2015, Colomina and Molina, 2014, Niethammer et al., 2012, Niedzielski et al., 2016] but, regarding the current legislative situation, constant autonomous observation (e.g. using scheduled flights) is not possible, particularly for urban river sections.In combination with photogrammetric approaches, laser scanning techniques have been proven successful for water level determination, i.e. by the application of laser light sheet triangulation [Mulsow et al., 2016].The vast majority of the mentioned approaches have one thing in common: data is acquired in the field but necessary information is derived in the office later.However, in case of flash floods, the time between the instrumental setup and final interpretation should be kept as brief as possible because river characteristics can change suddenly.Smartphones and modularly pocket computers, i.e.Raspberry Pi's, are able to acquire and process data, gain and visualise information, and furthermore, can transmit the results to the final users not necessarily present in the field.
The analysis of textural image features for image segmentation and boundary extraction, pointing to the shore line, is a common approach for image-based water line detection.However, each presentation of texture is highly correlated with perception.Thus, close-range images of running waters can show entirely different textural features in contrast to the appearance from a bird's-eye view, where running water seems to be very similar, making them classifiable [Verma, 2011, Martinao et al., 2003, Haralick et al., 1973].Time-lapse images provide an additional dimension, i.e. the third dimension time, to enhance the distinction of river lines.More details about this approach are given in [Kröhnert and Meichsner, 2017].
The retrieved water line has to be transfered into object space, which can be expressed as annotation, a term commonly used in computer graphics, of 3D data using registered 2D images [Schwalbe and Maas, 2017, Kehl et al., 2016, Chen et al., 2008].Popular use cases can be found in augmented reality (AR) where virtual image content is projected into the real world using mobile devices as interfaces, e.g.smartphones or AR glasses.In applied geo-sciences, [Kehl et al., 2017,Kehl et al., 2016] use an Android tablet for field-based geological interpretations in petroleum geology.Virtual renderings of available 3D base data are generated to solve the issue of field-based 2D-3D matching.Common approaches are mesh- [Sattler et al., 2011], splat- [García et al., 2015] or point-based rendering [Meierhold et al., 2010].If an virtual image with known pixel depths is available, the issue of pose estimation can be described as Point-n-Perspective (PnP) optimisation problem with additional camera geometry parameters [Lepetit et al., 2009, Marquardt, 1963].

LOW-COST MEASUREMENT HARDWARE
Prior discussing the hardware system, the term low-cost needs to be clarified.To put it in a nutshell, [Potsiou and Ioannidis, 2003] define low-cost as "good results within less time and low budget".Thus, smartphones equipped with inbuilt cameras, location sensors and powerful processing units offer stunning newly instruments in scientific research.Furthermore, single board computers, such as the Raspberry Pi, gain more and more popularity due to their modularly extensibility, tailored to the individual needs of the users, at low costs.

Smartphone
During the last decade there has been a significant change in terms of worldwide mobile smartphone subscriptions with still rising trends1 .Today's smartphones can be considered as small pocket computers instead of mobile phones with advanced sensors and cameras.They can serve as widely spread measurement devices for various applications also in geo-sciences.Beside the already mentioned scenarios in 1.1, common tasks comprise e.g.mobile mapping [Masiero et al., 2016, Verma et al., 2016, Westhead et al., 2013], object reconstruction with Structure from Motion (SfM) [Masiero et al., 2014] or, river monitoring [Pena-Haro  et al., 2015et al., , Kim et al., 2013]].Most of the smartphones provide sensors for orientation such as accelerometer, magnetometer and gravity meter and for position and navigation a Global Navigation Satellite System (GNSS) module.Increasingly, gyroscopes and barometers are installed to support orientation and relative positioning via sensor fusion [Dabove et al., 2015, Mourcou et al., 2015].Nevertheless, compared to commercial state of the art sensor systems, as used in autonomous navigation, the values obtained by smartphone sensors must be critically reviewed [Kröhnert et al., 2017].A smartphone can be a suitable device for photogrammetric measurements, using the inbuilt camera for data acquisition, location sensors to determine the exterior orientation parameters, and the computational power.Thus, smartphones allow for field-based research.

Raspberry Pi
Besides smartphone cameras, low-cost single board computers equipped with a simple camera circuit board can be used to observe river cross sections.In this study a Raspberry Pi computer and a Raspberry Pi camera v2.1 are utilised.Using such a board enables the opportunity of simple script implementations to trigger the camera in specified temporal sequences and capture videos with specific lengths.Furthermore, intelligent image capturing is possible, e.g.considering previously captured images and afterwards extracted information.

STUDY AREAS
For application-based development and evaluation of the methodology, three rivers with small-and medium-scale catchments, situated in Saxony, Germany (see table 1) are gauged.Stationary Raspberry Pi cameras are installed at the creek Triebenbach and the river Wesenitz.To evaluate on the fly water stage measurements using smartphones, an urban section of the river Weißeritz is observed.Here, measurements are taken twice each day (early in the morning and later in the afternoon) for seven days.The observed section became known through the media during the floods in 2002 and 2013 after heavy rainfalls (figure 1).All measurements at the three rivers are performed close to official gauges that serve as references (see section 6).

DATA
For field-based surface annotation, the image measurement must be transferred into object space to detect water levels.In order

Digital surface models
SfM is characterized as a very flexible, fast and easy to use technique to generate 3D data.In a nutshell, SfM can be considered as an enhanced method of conventional stereo photogrammetry that applies multi-view geometry allowing for the generation of coloured 3D representations of almost any-sized objects.
After image acquisition, feature points are detected and matched and subsequently used within the image bundle to reconstruct the camera configuration via bundle adjustment.The reconstructed camera geometry serves as basis for the posterior dense matching.Table 2 gives an overview of the 3D point surfaces in relation of each study area (see section 3). Figure 2 shows the multi-view geometry for the river Weißeritz.The reconstruction of river beds is difficult using SfM due to water cover.However, for shallow, calm waters being clear and almost translucent, it is possible to reconstruct the bed considering water refraction during depth estimation [Dietrich, 2017].
4.1.1(Geo) referencing A joint reference system has to be defined to retrieve information using data from different sources.Therefore, the underlying topography as well as the image data for registration, captured by the camera-gauges, have to be in a common reference frame.
For the fixed camera gauges, a local and scaled reference system is already sufficient and no geo-referencing is necessary.12 and 17 well-distributed temporary ground control points (GCPs) at the Triebenbach and Wesenitz, respectively, have been measured with a total station providing accuracies of a few millimetres.Additionally, eight stones serve as permanent GCPs close to the shore line of the Wesenitz.GCPs are further used for point cloud alignment of the TLS-and SfM-based point clouds utilizing the advantages of both data sources [Eltner et al., 2016].
Mobile water gauging aims to be used in any situation where 3D representations of river sections are available (e.g.3D point clouds or DSMs).Thus, the surface data needs to be available in a global reference frame within which the smartphone can be situated.At the Weißeritz, the SfM-derived point cloud has been geo-referenced using six survey points and four scales (figure 2) to apply a simple Helmert transformation.The overall root mean square error (RMSE) is 4cm and residuals of 2.5cm can be observed at GCPs close to the shore line.
4.1.2Reference water stage measurements Currently, official gauging stations provide accuracies of ✟1cm for stage measurements, averaged over a defined time interval [Siedschlag, 2015].[Horner et al., 2018] divide measurement error sources in stage reading errors as well as non-systematic error sources that are negligible and in systematic errors that are in the range of 0.5cm ✁ 6.8cm and are caused by sensor drift or invalid calibration.Thus, especially uncalibrated stage values from conventional water gauges should be regarded critically.Furthermore, it is important to acknowledge that comparing water stage measurements can cause bias simply due to the impact of varying temporal resolutions of the individual measuring devices.
Camera-based water measurements are performed close to conventional gauges to allow for reliable reference data and evaluation of the performance of the camera-gauges.

Camera requirements
The camera has to be adapted regarding its intrinsic and extrinsic parameters to ensure the precise alignment of camera images and 3D topography of the area of interest.This section discusses the individual requirements of mobile (see section 4.2.1) and stationary camera calibration (see section 4.2.2) considering the individual study areas.

Smartphone camera considerations
The approach of mobile water gauging aims at the citizen scientists, who would like to participate in the densification of a hydrological network simply by taking images with their smartphones.Consequently, a simple camera application is necessary that supports hand-held time lapse images and 2D water line detection.At the same time, the application has to be able to collect information about the camera position and orientation to reference the image measurements which is further described in section 5.

Raspberry Pi camera considerations
The low-cost camera used to observe the Triebenbach and Wesenitz are located close to the rivers at a tree and a lamp pole, respectively, at least 2 m above the water surface.The cameras have a fixed focal length about 3 mm and are focused at infinity.Pixel size is 1.4 µm and image resolution amounts 2592 x 1944 pixels.To allow for the operation at remote locations, the camera system at the Triebenbach is equipped with a car battery.To guarantee enduring operation, an energy saving system is added to the Pi, which boots the single board computer every hour for a few minutes and puts the system back into sleep mode until the next scheduled boot.At the Wesenitz permanent power supply is given.Hence, the computer runs permanently and solely a light sensor controls image capturing to avoid data collection during the night.
The simple circuit cameras entail the advantage that they are quite robust regarding their interior camera stability due to the usage of solely few elements, which are integrated in a fixed manner.
Prior or posterior time-lapse image acquisition at the river cross sections, the cameras have been calibrated with a temporary calibration field that has been shifted, tilted, and rotated in-front of the cameras.Afterwards, a bundle adjustment is performed to retrieve focal length, principle point and distortion parameters.The parameters are used for subsequent image measurements assuming temporal stable interior camera configurations.

APPLICATION DEVELOPMENT
On the one hand, the chapter discusses the methodology to retrieve water levels from image data and, on the other hand, it provides insights into the technical implementation of smartphones, i.e. the Android application Open Water Levels is introdcued, which is based on the open source framework Open Camera v1.3.84 .

Time lapse image sequences for water line determination
Even fixed outdoor cameras are affected by camera movements due to environmental influences, e.g.wind, resulting in changes of the calibrated exterior camera configuration.Of course, similar issues occur during hand-held time lapse image acquisition using smartphone cameras.However, in the context of spatio-temporal texture analysis, a non-rigid camera leads to considerable noise, which impedes water line derivation.
A possible solution is provided using co-registration of the time lapse images with a feature-based matching approach implementing a subsequent perspective transform.Afterwards, the image becomes divisible due to the captured motion of water.Thereby, multi-seeded region growing is applied within a user-defined region of interest (RoI) using either the spatio-temporal texture or the average image depending on the distinctiveness between water and land in both data sources.Once the image is classified in rigid and non-rigid parts, i.e shore and water areas, the shared boundary is identified as water line.Detailed descriptions of the approach are given in [Eltner et al., under review, Kröhnert and Meichsner, 2017].

Surface annotation
Once image measurements are performed, they have to be transferred into object space to retrieve actual information about the water level.Considering section 4.2.2 and 4.1.1,the whole setup of the fixed low-cost solution (Pi camera), i.e. interior and exterior camera parameters, can be calibrated in advance and solely minor changes are adjusted using image co-registration.
However, the mobile approach has to be location-independent to be suitable for citizen scientists, making it necessary to estimate camera parameters on the fly, i.e. using self-calibration, without registration targets.

Scene description using smartphones
Global referenced object data serves as a big cloud of GCPs that has to be assigned to the corresponding time lapse image sequence.A virtual image is rendered that is based on initial information about camera position and orientation.Open Water Levels analyses the device-specific hardware sensors to consider the best approach to orient the data whereas the inbuilt single-frequency receiver is called to estimate the pose.Absolute GPS (aGPS) of smartphones provide accuracies in the range of 10 to 15 m and can increase up to 60 m due to shading effects [Blum et al., 2013].Furthermore, it is important to consider that the error for altitude estimation is nearly three times worse than for lateral pose estimation [Liu et al., 2014].Open Water Levels offers the possibility to include third party tools like Google Maps or Google Elevation via application programming interfaces (APIs) to allow for user-driven pose estimation and height optimization using the additional digital elevation models (DEMs).

Generation of artificial images
A set of coloured 3D points, which are maintained in a central database, has to be chosen considering the initial guess of extrinsic parameters before starting the actual rendering.A bounding box can be calculated with a defined depth from the projection center if camera position and orientation parameters describing the uncertainties of pose estimation are utilised.
Virtual images are generated projecting the 3D point cloud at a defined image plane.Thereby, the method is enhanced using a pyramid approach, reducing artefacts due to falsely projected obscured points (e.g. points behind windows or opposite facades).
Although the virtual rendering serves as two-dimensional representation of the object scene generated from a specific perspective, real depth information is still present.Thus, using the projected image points and their original 3D representation, remaining gaps due to point cloud sparsity and virtual image resolution can be filled to generate a realistic visualisation.A nearest neighbour search is applied in object space, using the 3D information of image points located in close proximity to the gap to retrieve colors with a weighted average constraint by the neighbour distances.Figure 2 shows a rendered object scene at the Weißeritz, using an initial guess of exterior orientation captured during image acquisition (see figure 4).

Image-to-geometry registration
The original image and the virtually generated image are aligned in the last processing step, allowing for exterior camera geometry estimation to enable the determination of water levels.Open Water Levels establishes an online connection to enable server-based image-togeometry registration.Afterwards, a small file archive is generated containing the time lapse master image, a text file with the image coordinates of the derived water line and a JavaScript Object Notation (JSON) file containing camera "meta" information (f.e. the sensor-based rotation matrix and coordinates of the projection center obtained by GNSS as well as intrinsic information like the focal length or the view angles).If no server connection is possible, the entire 3D data needs to be downloaded on the device [Kehl et al., 2017].Figure 3 illustrates the client-server architecture [Niemann, 1995].
Due to the use of true 3D data in virtual image rendering, feature The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020", 4-7 June 2018, Riva del Garda, Italy  based matching approaches are applicable also to align true and synthetic images and thus allowing for the usage of true images from the 3D point set.SiftGPU, implemented in VisualSFM, is running on a remote working station for feature detection, description and matching.Outliers are detected applying fundamental matrix calculation in combination with Random Sampling Consensus (RANSAC).A synthetic image contains several gap filling pixels that do not originate from true 3D points.Thus, only matched features with assigned information in the spatial domain are considered when a Direct Linear Transform (DLT) is used to estimate the exterior camera parameters.This approach can be described as a non-linear PnP problem, optimized via the Levenberg-Marquardt algorithm that is implemented in OpenCV 3.0.0[Kaehler and Bradski, 2016].Thereby, intrinsic camera parameters are adjusted, minimizing the re-projection error of the individual feature points.For an initial guess of camera parameters, the focal length of the fixed lens is used, which is specified by the manufacturer.Furthermore, the principle point as well as the lens distortion are assumed to be zero.In a last step, the water line is projected into object space using the optimised parameters.Thereby, a knn search, implementing distance thresholding, is applied to find the closest points in the spatial domain for each point of the measured 2D water line.To obtain a single value for the prevalent water stage, either mean, or median heights are used.Prior mean or median calculation, a 3D line fit is performed to eliminate potential outliers.Finally, feedback about the stage measurement are offered to the client.
The Raspberry Pi implementation does not rely on the calculation of virtual images due to the referencing within a local reference system, which is sufficient for water level estimation at predefined stable gauge locations.Thus, section 5.2.1 and 5.2.2 are not considered for the single board computer cameras.However, image to object space transformations are also performed for the fixed low-cost cameras at Triebenbach and Wesenitz to retrieve the filtered river stage information from the 2D waterline.

EVALUATION
As described above, water levels, captured at the reference gauging stations, are used to evaluate the camera-based water stage measurements of both approaches (smartphone and simple camera circuit board).Reference data is available every 15 minutes for 24 hours.
Using the smartphone-based approach, the Weißeritz was monitored for one week two times per day, i.e. in the morning and in the afternoon.Regarding the stationary observed rivers Triebenbach and Wesenitz, water stages are obtained every 60 minutes and every 30 minutes, respectively, for at least six weeks, as long as day light was sufficient.At the Weißeritz, the smartphonegauging period was characterised by events of low precipitation and thus low flow conditions.Figure 5 shows the deviations of conventional and camera-based gauging systems based on the median.Thereby, accuracies higher than 5 mm and 1 cm could be achieved at the Triebenbach and Wesenitz, respectively.Furthermore, the median deviation is less than 3 cm at the Weißeritz using the smartphone-based approach.
image-based water line estimation Some of the water levels estimated with the image-based approach are associated with the shore area due to too low spatio-temporal resolution.Furthermore, vegetation covering the shore, waving or swimming grasses, and sensor-related image noise cause textural noise, which impedes a precise water line analysis.Beside these nonsystematic error sources, the resolved water line is highly correlated with image resolution and object to camera distance.Thus, the water level might be systematically underestimated especially if shallow water is very close to the shore.In contrast, conventional measurement systems are installed with a setup that minimizes the influence of the water surface and environmental factors.During the short observation periods, waves close to the shore often result in water stages being overestimated with the camera-based approach.
smartphone-based gauging Unlike the Raspberry Pi gauging systems, the relationship between smartphone image space and object space has to be established prior the water line can be calculated.Regarding the registration of an image in the object space, utilized feature matches need to be well-distributed in image and object space.Otherwise, the adjustment of exterior and interior camera parameters might be adjusted at a false local extreme value.Furthermore, features solely lying on a planar surface can result in an erroneous focal length estimation and thus scale-dependent errors in object space.

OUTLOOK
The paper demonstrates the successful implemenatation of camera-based water stage measurement systems to perform stationary and on the fly water gauging.The introduced approach requires 3D surface models or point clouds, of the observed river reach or cross section, which can hinder the usage of the method.However, Google Street View recently performed first river cruises to cover the shores of River Negro in Amazon, Brazil5 , that can be globally expanded to capture river morphologies and thus be available for the image-based water level calculations.Furthermore, autonomous river mapping tools are developed in-house to capture small-and medium-scale catchments, also providing the necessary data for camera-gauges [Sardemann et al., 2018].

Figure 2 .
Figure 2. Study region Weißeritz in Dresden, Altplauen: SfM configuration with 3D point cloud overlay (left) and detail shot (right) close to a official gauge.

Figure 3 .
Figure 3. Workflow of event-based camera gauging using smartphones.

Table 1 .
Study areas to obtain three-dimensional reference data, several approaches are conceivable.In this study, SfM is applied to generate 3D point sets from terrestrial images or aerial images captured with an UAV.Additionally, Terrestrial Laser Scanning (TLS) was applied at the Triebenbach and Wesenitz to check the quality of the SfM-based point clouds.
Table 1 gives an Table 2. Generation of base topography by application of SfM.
*low, mid, high water tides overview of the installed systems with respect to the study areas Weißeritz, Wesenitz and Triebenbach.