TOWARDS A LOW-COST , REAL-TIME PHOTOGRAMMETRIC LANDSLIDE MONITORING SYSTEM UTILISING MOBILE AND CLOUD COMPUTING TECHNOLOGY

Close-range photogrammetric techniques offer a potentially low-cost approach in terms of implementation and operation for initial assessment and monitoring of landslide processes over small areas. In particular, the Structure-from-Motion (SfM) pipeline is now extensively used to help overcome many constraints of traditional digital photogrammetry, offering increased user-friendliness to nonexperts, as well as lower costs. However, a landslide monitoring approach based on the SfM technique also presents some potential drawbacks due to the difficulty in managing and processing a large volume of data in real-time. This research addresses the aforementioned issues by attempting to combine a mobile device with cloud computing technology to develop a photogrammetric measurement solution as part of a monitoring system for landslide hazard analysis. The research presented here focusses on (i) the development of an Android mobile application; (ii) the implementation of SfM-based open-source software in the Amazon cloud computing web service, and (iii) performance assessment through a simulated environment using data collected at a recognized landslide test site in North Yorkshire, UK. Whilst the landslide monitoring mobile application is under development, this paper describes experiments carried out to ensure effective performance of the system in the future. Investigations presented here describe the initial assessment of a cloud-implemented approach, which is developed around the well-known VisualSFM algorithm. Results are compared to point clouds obtained from alternative SfM 3D reconstruction approaches considering a commercial software solution (Agisoft PhotoScan) and a web-based system (Autodesk 123D Catch). Investigations demonstrate that the cloud-based photogrammetric measurement system is capable of providing results of centimeter-level accuracy, evidencing its potential to provide an effective approach for quantifying and analyzing landslide hazard at a local-scale.


INTRODUCTION AND BACKGROUND
Landslides are one of the most commonly-occurring natural phenomena worldwide (Davies, 2015).Such events not only pose a serious threat to human life and society, but often lead to significant economic losses (Regmi et al., 2015).It is therefore essential to develop systems that are able to monitor landslide hazards in a timely manner.Moreover, the development of such systems may also help to enhance understanding of landslide behaviour.Ground-based landslide monitoring approaches are generally implemented utilising geotechnical or/and geophysical techniques such as piezometers, pore pressure sensors, inclinometers, and electrical resistivity tomography.Geomatics techniques can also be used to measure surface movements of landslides, for example Global Navigation Satellite System (GNSS), satellite remote sensing, and photogrammetry.Such techniques can be split into two main groups of landslide monitoring: airborne/space-borne and ground-based approaches, both of which can be used to investigate landslide kinetics, consisting of ground movements and displacement rates.The main advantage of the first approach is that it is generally less labour-intensive than conventional techniques or traditional surveying.However, space-and airborne-based monitoring approaches, e.g.InSAR, lidar, aerial photogrammetry or UAV platforms, are generally more suitable for landslide detection over large areas (Niethammer et al., 2012).Adoption of such * Corresponding author approaches for small area monitoring may not be cost-effective, particularly when data needs to be frequently captured (Scaioni et al., 2014).
For small landslide features, geotechnical and geophysical methods are well-established and offer effective acquisition of sub-surface information (Uhlemann et al., 2016).Despite their suitability for landslide monitoring, their main limitation is the discrete character of the observations and their restricted spatial coverage.The establishment of geotechnical and geophysical monitoring systems can also be labour intensive, invasive and not always cost-effective.Consequently, installation of such systems is often more suitable once a landslide is known to exist (Perrone et al., 2014).However, sub-surface processes of landslides often only reveal themselves through surface expression of movement.In contrast, surface observations using geomatics techniques can play an important role in low-cost monitoring for landslide assessment because surface deformations will often reveal underlying patterns of failure (Miller et al., 2008).In particular, ground-based geomatics techniques, including terrestrial laser scanning (TLS), ground-based synthetic aperture radar interferometry (GB-InSAR) and close-range photogrammetry all offer non-contact monitoring and a spatially-continuous surface representation.
TLS is able to deliver high resolution, high accuracy point clouds which can be extremely valuable for detailed landslide assessment (Scaioni, 2015).However, TLS equipment is costly, requires expertise in operation, and collection times can be relatively slow, particularly if data must be collected from multiple locations.Furthermore, data gaps can occur in landslide areas due to oblique perspective and terrain occlusions (Teza et al., 2007).An alternative approach is GB-InSAR which, similarly to TLS, can be operated at day or night.GB-InSAR provides the measurement of displacements, but these observations must be corrected for topography, necessitating the collection of further reference data (Bardi et al., 2014).Moreover, atmospheric effects can affect the quality of measurements (Bozzano et al., 2011).Consequently, GB-InSAR is only suitable for application by expert users, is costly, and requires significant post-processing to derive meaningful deformation measurements.Photogrammetric approaches have been widely used for landslide monitoring (Akca, 2013;Stumpf et al., 2015).In particular, close-range photogrammetry can provide high measurement precision, potentially from millimeters to a few centimeters at 100 m range, and in this sense is comparable to TLS and GB-InSAR.In addition, photogrammetry offers instantaneous data capture, and in comparison to TLS and GB-InSAR, the costs associated with instrumentation and maintenance are considerably lower (Travelletti et al., 2012).Close-range photogrammetry is thus an attractive approach for assessment of landslide process.
The development of Structure-from-Motion (SfM) techniques has improved the accessibility of photogrammetric processes for use by non-experts, and increased automation (Javernick et al., 2014;Westoby et al., 2012).At the same time, it has been shown that the quality of results can conform to expected levels of accuracy for conventional photogrammetric processing (Micheletti et al., 2015).However, in order to achieve high accuracy measurement using the SfM approach for geoscience applications, there are many factors which must be considered.For example, the use of a digital single lens reflex (DSLR) camera for image acquisition delivers high resolution and high quality imagery for photogrammetric measurement (James and Robson, 2012).In addition, a large number of overlapping images are required to fill data gaps for deformation monitoring.Consequently, this results in an extremely time-consuming SfM process due to the large volume of data to be processed.Clearly, such considerations do not always conform to expectations for using the SfM-photogrammetric approach in order to monitor landslide hazards in a timely manner.
Nowadays, there are an ever-increasing number of affordable mobile devices, such as tablets and smart phones, on the market.Such mobile devices generally employ in-built, high-resolution digital cameras that offer significant potential as low-cost sensors for close-range photogrammetric monitoring (Micheletti et al., 2015;Wang et al., 2012;Yun et al., 2012).However, one current drawback of mobile devices is their low-performance computing power when compared to both personal and laptop computers.This presents a major challenge for data-intensive processing on a mobile device.However, internet services via 3G/4G networks and cloud computing technology have developed rapidly, and can now be considered relatively standard.This has opened up the potential for harnessing additional processing power, through the implementation and deployment of geoinformatics applications on mobile cloud computing technology (Lee and Kang, 2013).Thus, the latest technology provides the possibility of development of real-time measurement systems using closerange photogrammetric techniques for landslide monitoring using mobile devices.
The research reported herein aims to develop a mobile application for real-time photogrammetric landslide monitoring based on cloud computing technology.Freely available open source SfM software, coupled with a mobile phone camera sensor, enables implementation of a photogrammetric measurement system through a low-cost approach.Moreover, the optimisation of a close-range photogrammetric approach based on the SfM technique and cloud computing technology is expected to reduce the constraint of data processing time that, todate, has been a significant challenge in real-time assessment.While the mobile application for landslide monitoring is still under development, this study is aimed at demonstrating the potential of such a system by testing its components.Experiments are carried out using data collected at a recognized landslide test site to determine the accuracy and reliability of the results under simulated conditions.Further, the capabilities of such a low-cost system for dynamic monitoring applications are assessed by the means of comparison with the results produced by other commercial and non-commercial software.
The remainder of this paper is organised as follows: the study site and datasets that were used in this research are described in Section 2. Section 3 outlines the development of a photogrammetric measurement system on the cloud, as well as the experiments carried out to ensure effective performance of the system.Section 4 presents the SfM results and evaluation of the system components compared to two other alternative SfM methods.Finally, Section 5 focuses on the next stages of development of the mobile application for landslide monitoring.

Study area
The study area is located at the British Geological Survey (BGS) Hollin Hill landslide observatory in North Yorkshire, UK (Figure 1).A number of in-situ monitoring systems are already installed at the site, including those based on geotechnical and/or geophysical techniques, such as electrical resistivity tomography (ERT) which is used for real-time monitoring (Merritt et al., 2014).The Hollin Hill landslide is characterised by a shallow rotational failure at the top of the slope, which then moves through an area of translational landslide movement at the middle of the slope.The landslide extends as flow lobes towards the bottom of the slope.The landslide is mostly caused by the movement of the Whitby mudstone formation over the Cleveland ironstone and Staithes sandstone formations, which is highly prone to land sliding.The Hollin Hill landslide is vegetated with short grass, which is pasture land for sheep.The focus of this study was the main scarp at the top of the slope (red box in Figure 1), which extends for 25 m in an approximately east-west direction, and 25 m from north to south, with 8 m elevation difference between the top and bottom of the scarp.

Datasets
To achieve the necessary quality of results from a close-range SfM approach, it is necessary to consider the photogrammetric network design (Luhmann et al., 2006).The imaging plan is calculated by four main factors as follows: (1) base to depth (B/D) ratio of imaging geometry must be in the range of 0.1-0.3 to provide accurate ray intersection for 3D reconstruction (Hullo et al., 2009;Waldhäusl and Ogleby, 1994); (2) the maximum distance between the camera and the object must be considered (in this experiment it was kept at less than 30 m) because image scale, a function of the focal length of the camera lens, has a direct impact on the measurement precision; (3) a camera field of view (FOV) of ~40° to 80° is generally required in deformation monitoring for engineering applications (Fryer et al., 2007), which is applicable here, given the requirement to capture detailed surface information; (4) images should fully cover the study area (360° coverage), with necessary overlaps, to enable effective 3D reconstruction.
Imagery was captured on 10 June 2015 using a Nexus 6 mobile phone camera with a built-in 3.82 mm f 2.0 lens.This comprised 36 images with a maximum image size of 4160 x 3120 pixels (approximately 13 MP).Camera stations and viewing direction are shown in Figure 2. Markers, established using six photo control targets printed on paper and located around the landslide were used for georeferencing of the photogrammetric results.
Additionally, for validation assessment, results were compared with a high-resolution dataset collected on the same date using a terrestrial laser scanner (TLS).TLS data collection was performed using a Leica ScanStation P20.In order to avoid data gaps over landslide areas, the scene was captured from four scanner positions.Finally, the locations of the six photo control targets and the four TLS stations was observed using Global Navigation Satellite System (GNSS) at millimeter-level accuracy.
Figure 2. Camera positions for photogrammetric image capture.

System development
A mobile device (in this instance a smartphone) and cloud computing services are used for the development of a photogrammetric measurement and landslide monitoring system.The development of the system is divided into two main aspects: (1) a remote, cloud-based server that is used to store and process image data in order to generate the photogrammetric results; (2) a local client, running on the mobile device, is used to upload images to the cloud and then subsequently display the photogrammetric results (such as 3D point clouds of the landslide area).The system is based on client-server communication via the Hyper Text Transfer Protocol (HTTP) using the Internet service, as shown in Figure 3.The server-side of the system adopts the Amazon Elastic Compute Cloud (Amazon EC2) (Amazon Web Services, 2016) as the cloud computing service and is used for the processing of the photogrammetric data.The system is implemented on a graphical processing unit (GPU) server in the g2.2xlargeAmazon EC2 instance type in order to handle the high processing demands of the SfM software.The GPU server implements several software routines.Firstly, jQuery File Upload is used to upload image files from the mobile device to the server.Secondly, VisualSFM (Wu, 2011), freeware based on the SfM technique, is used for 3D reconstruction of the mobile phone imagery.The photogrammetric results obtained from VisualSFM are generated as point clouds.Thirdly, the PotreeConverter open source code (Schütz, 2015) is used to convert the results to Potree format to facilitate rendering of the point clouds through a Web based 3D viewer.Fourthly, the results are transferred from Amazon EC2 and stored in the Amazon Simple Storage Service (Amazon S3), a cloud storage service using the AWS Command Line Interface (CLI) in order to avoid losing data.Finally, the back-end service of the system uses a PHP script in order to control the workflow of photogrammetric measurement system on the server.

Server
An Android mobile application, based on the client-side of the system, is currently under development for implementation on a mobile platform.The application will be used as the front-end viewing platform for the system and will also help to connect to the landslide monitoring system based on the cloud in order to upload images, send data processing requests and display the outputs of processing.This mobile application, named Sky Photogrammetric Measurement and Monitoring System or SkyPMMS, is being developed using the Java programming language in the Android Studio integrated development environment (IDE).SkyPMMS will employ the thin client approach through HTML5 technology such that it can operate using a standard web browser on multiple platform types (e.g.tablet, desktop computer).

Experimental design
Since SkyPMMS has not yet been fully developed and implemented in real-world conditions, laboratory experiments have been carried out to assess the performance of its components using data collected at the Hollin Hill landslide site.Figure 4 shows the design of the experiment aimed at evaluating cloudimplemented VisualSFM performance in comparison to that of Agisoft PhotoScan (Agisoft, 2016) and Autodesk Catch (Autodesk, 2016), popular alternative SfM 3D reconstruction software approaches.PhotoScan is a commercial SfM software that is widely used for 3D scene reconstruction from both groundand air-based approaches.PhotoScan runs and processes data locally on a PC.123D Catch is an on-line SfM system whereby data is uploaded to a website and processed through a web service.It also offers fully-automated processing and is free to use.
Figure 4. Methodology flowline adopted in this research.
All SfM approaches used the same 36 image dataset (at 4160 x 3120 pixels of a maximum image size) from the Nexus 6 mobile phone camera.One of reasons why a development version of SkyPMMS could not yet be adequately assessed under field conditions was the fact that the networking performance of the mobile phone technology is strongly dependent on the speed of the Internet service.Thus, the 3G network available at the test site is not ideal for data transfer to the cloud.Unfortunately, 4G networks in the UK are still largely limited to towns and cities and are not yet available at the Hollin Hill study site.It is, however, expected that 4G will become more widely available in the future making data transfer less troublesome.For the above reasons the photographs taken in the field were transferred from the mobile capture device to the cloud using a Wi-Fi network under laboratory conditions.
Data processing in PhotoScan was carried out on a desktop computer running on Windows 8 with an Intel Core i7-4770 Processor, 3.4 GHz CPU, 16 GB of RAM and an Intel HD Graphics 4600 card, 1.4 GHz GPU.Cloud-implemented VisualSFM was running on the g2.2xlarge of Amazon EC2 instance under Windows Server 2012 with Intel Xeon E5-2670 Processor, 15 GB RAM and NVIDIA GPUs with 4GB of video memory.As Autodesk 123D Catch is a web-based black box SfM tool, the specification of the processing system is unknown and there are some limitations of usage that might affect the quality of SfM results.
The aforementioned six photo control targets, precisely measured by GNSS, enabled straightforward comparison of the results for each SfM approach in a common coordinate system.The results of VisualSFM implemented on the cloud, and Agisoft PhotoScan did not need post-processing georeferencing, as targets were observed directly in the imagery, and georeferencing was undertaken as part of the respective workflow.This part of the methodology required manual input by the user.In contrast, the SfM output from 123D Catch was not directly georeferenced (it is not possible to measure targets directly in the photos), and hence this had to be undertaken as a separate step.In this instance, georeferencing was carried out manually by identification of photo control targets in the point cloud using the CloudCompare software (CloudCompare, 2015).The use of photo control targets was necessary for georeferencing in order to provide the results of each SfM approach in a common coordinate system.In the next stage of development, it is intended that photogrammetric measurements will be performed without photo control targets, as direct contact with a landslide body is potentially hazardous and should be avoided if at all possible.
Due to repetitive texture patterns or poor image contrast, SfM is prone to matching errors in the multi-view stereo, dense surface reconstruction process (Scaioni, 2015).This is particularly so over vegetated surfaces such as those found in the landslide study area, and these can have a direct impact on the quality of results.In this study, matching errors were reduced by using an automatic de-noising filter based on a statistical outlier removal in CloudCompare.The de-noising step was used to improve the quality of the photogrammetric results before the evaluation of SfM results from all three approaches; it, however, failed to completely remove all gross errors, with remaining outliers removed manually.Finally, the comparison of photogrammetric results between SfM models and TLS data was carried out using the cloud-to-mesh distance tool in CloudCompare, thereby allowing evaluation of the SfM results.

Comparison of data processing
The data collection using TLS usually takes longer than a photogrammetric approach due to the nature of laser-based data capture.Moreover, the post-processing of TLS data can also be complex and normally involves significant manual interaction by an expert user.In contrast, SfM data collection is considerably faster than TLS, and the SfM method is arguably also better Evaluation suited to automation.However, the pre-and post-processing of the SfM approach does still require manual interaction, especially the georeferencing and de-noising stages.
This study used 36 image files that required uploading from a mobile phone to the cloud server.The total file size of this image set was approximately 260 MB.Wi-Fi with average upload speed of 40 Mbps was used to transfer the images from a mobile phone to the cloud, with the total upload time taking approximately 1 minute.As seen in Table 1, which summarizes the predicted upload time for such a dataset using different Internet networks, transfer using a 3G network would take approximately 35 minutes, which is clearly unsuitable for a real-time monitoring system.It is, however, anticipated that faster 4G internet will be more widespread in the future and would allow a near real-time response for such a system.

Type of mobile networks
Average upload speed (Mbps)

Photogrammetric results and comparisons
Visual comparison of the results provided by different SfM methods (shown in Figure 5) reveals that Agisoft PhotoScan was able to produce a much denser point cloud than both Autodesk 123D Catch and the cloud-implemented VisualSFM.Moreover, 123D Catch produced more uniform coverage than VisualSFM, which was sparser, especially over vegetated surfaces.Nevertheless, the resultant point clouds generated by the latter two methods still provided the key information over landslide areas, especially fissures and cracks on the landslide body.Using PhotoScan is not suited to on-site investigation of landslides because the software supports only a stand-alone system.On the other hand, Autodesk 123D Catch provides access for Internetbased processing, in a similar manner to cloud-implemented VisualSFM.However (depending on the application), manual target referencing in the point cloud (123D Catch) proves more difficult than target identification in images (VisualSFM), which makes the process less convenient for the end user.The intention is that the fully-developed SkyPMMS mobile application will offer a fully-automated data processing workflow for on-site photogrammetric measurement compared to the other SfM methods.
The TLS data was used to evaluate the photogrammetric results from each SfM method.Due to the range of surface cover of the landslide area, which includes vegetation, the acquired TLS data was regarded as a digital surface model (DSM).The SfM approach also generates DSMs, as photogrammetric approaches are only able to capture the visible surface, and hence are unable to penetrate vegetated surfaces.To evaluate the results, each SfM DSM was subtracted from the TLS DSM using the cloud-to-mesh distance tool in CloudCompare.The statistics of the comparison between each SfM DSM and the TLS-derived DSM are presented in Table 2.In addition, the distribution of DSM elevation offsets are shown in Figure 6.Table 2 shows that the range of elevation differences between PhotoScan and the TLS reference was noticeably higher than that of VisualSFM and 123D Catch.This is due to the much denser point cloud produced by PhotoScan which consequently means that the vegetated surfaces have greater impact on the range of  Based on the presented results, the means of the elevation differences between the TLS DSM and the SfM DSMs all show positive values, indicating that all SfM DSMs were slightly above the TLS DSM.The standard deviations of the differences between the TLS DSM and the SfM DSMs obtained from PhotoScan, VisualSFM and 123D Catch were 0.057, 0.070 and 0.070 m, respectively.Consequently, the photogrammetric approach using the SfM technique can be deemed to provide results at centimetre-level accuracy for this type of landslide surface.

CONCLUSIONS AND FUTURE WORK
This research has shown the potential of mobile cloud computing that exploits modern information and communication technology in order to provide a photogrammetric measurement solution on a mobile device for the purpose of landslide monitoring.In particular, this approach supports off-the-shelf hardware (including affordable smartphone cameras) and open source software in order to deliver a low-cost monitoring system.Although SkyPMMS is not yet fully-developed and operational, the presented experiments utilizing cloud-implemented VisualSFM show the substantial potential of such a system for landslide monitoring over limited spatial extents.
Data processing on the cloud depends on the computing performance, especially the speed of both the CPU and GPU on the cloud server.The developed system has not yet taken into account a solution for real-time processing.In the next stages of the research, to enable a real-time response, the image upload and processing time are crucial factors.Therefore, optimisation of the imaging network, by optimising both the number of images and the image size, will be investigated.Subsequently, multi-epoch analysis will be considered in order to assess the photogrammetric results over time, thereby yielding information on landslide dynamics and hazard assessment.

Figure 1 .
Figure 1.Overview of test site located at Hollin Hill landslide observatory: the red boxed area is used for this research.

Figure 3 .
Figure 3. Overview of the mobile and cloud computing photogrammetric measurement system.

Figure 6 .
Figure 6.Normal distribution of elevation difference between each SfM result and DSM of the TLS data.

Table 1 .
Estimated data transfer time from a mobile phone to the cloud server for 36 images (260MB in total).

Table 2 .
Statistics of elevation differences between the different SfM DSMs and the TLS DSM.