APPLICATION OF SAR TIME-SERIES AND DEEP LEARNING FOR ESTIMATING LANDSLIDE OCCURRENCE TIME

The time series of normalized difference vegetation index (NDVI) and interferometric coherence extracted from optical and Synthetic Aperture Radar (SAR) images, respectively, have strong responses to sudden landslide failures in vegetated regions, which is expressed by a sudden increase or decrease in the values of NDVI and coherence. Compared with optical sensors, SAR sensors are not affected by cloud and daylight conditions and can detect the occurrence time of failure in near real-time. The purpose of this paper is to automatically determine the time of failure occurrence using time series coherence values. We propose, based on some existing anomaly detection algorithms, a deep neural network-based anomaly detection strategy that combines supervised and unsupervised learning without a priori knowledge about failure time. Our experiment using July 21, 2020 Shaziba landslide in China shows that in comparison to widely used unsupervised methodology, the use of our algorithm leads to a more accurate detection of the timing of the landslide failure. * Corresponding author


INTRODUCTION
Along with rising global temperatures, extreme weather events such as intensifying cyclones, hurricanes, and heavy rains are on the rise on a global scale, as well as the frequency of landslide occurrence in many parts of the world. Landslides can mobilize large amounts of material and cause significant damage to local populations, infrastructure, and buildings (Cao et al., 2018;Motagh et al., 2013;Petley, 2012;Schlögel et al., 2015;Wasowski and Bovenga, 2014;Xia et al., 2022). Timely generation of information on the temporal and spatial distribution of landslides will help in risk preparedness and emergency response management.
Precise detection of landslide occurrence time is a big challenge for landslide evolution research. Optical and synthetic aperture radar (SAR) satellite data are increasingly used to support landslide occurrence detection due to their multi-spectral and textural characteristics, multi-temporal revisit rates, and large area coverage (Behling et al., 2016). Landslide-prone areas are usually exposed to rainfall and thus, are often covered by clouds, which limits the use of optical images. Due to the cloud penetration capability of SAR sensors, more precise temporal characterization of landslide occurrence on a regional scale is possible using SAR satellite data. The Copernicus Sentinel-1 satellite mission consists of two satellites enabling a revisit cycle of 6 days while covering most of the world's landmass. These data form the basis for a more accurate detection of the timing of the landslide failure. Following the failure in vegetated areas, the coherence time series in the landslide regions suddenly increases and keeps high values for a certain period of time. Therefore, we can use the abrupt change in coherence time series, in response to the occurrence of failure, to obtain the time of failure.
For the detection of anomalies in the coherence time series, the currently popular unsupervised time series anomaly detection algorithms can be used. These include: clustering-based models, probabilistic-based models, LOESS (locally estimated scatterplot smoothing) models, prediction-based models, statistical models, and algorithms based on machine learning and deep learning (Goldstein and Uchida, 2016;Munir et al., 2019;Teng, 2010). K-means (Münz et al., 2007), Isolate Forest (Liu et al., 2008), Auto regressive Integrated Moving Average model (ARIMA) (Zhang, 2003), Seasonal and Trend decomposition using Loess (STL) (Robert B Cleveland et al., 1990), Autoencoders (Sakurada and Yairi, 2014), and Breakout detection (James et al., 2016) are among the popular approaches for these types of anomaly algorithms. In this study, we propose an anomaly detection strategy that combines unsupervised learning-based Breakout detection with supervised learning-based deep learning LSTM to automatically detect the time of failure occurrence using SAR coherence time series.

METHOD
In this section, firstly, selected time series anomaly detection methods are briefly introduced. K-means, Isolate Forest, ARIMA, STL, Autoencoders, and Breakout detection anomaly detection methods are chosen to be applied for the July 21, 2020 Shaziba landslide occurrence time detection. Then, we will discuss the advantages and shortcomings of different methods in anomaly detection. Finally, we propose our deep neural network-based time series anomaly detection strategy for detecting landslide occurrence time.

Time series anomaly detection methods
Time series anomaly detection methods are mainly classified as supervised, semi-supervised, and unsupervised detection. This study requires the detection of accident occurrence time without a priori knowledge, so we only focus on widely used unsupervised approaches as detailed below.
2.1.1 K-means: K-means is a typical clustering-based anomaly detection method (Münz et al., 2007). The principle of K-means is not complicated. First, several initial clustering center points are randomly set. Then each data point is assigned to the nearest cluster center point. After the assignment is completed, the average of the distances from the points in each class to the center point of that class is recalculated. If the cluster center has changed compared to the previous cluster center, each data point is reassigned to its nearest cluster center again. The iterations are continued until the cluster center does not change or the maximum number of iterations is reached.

STL:
STL is a classical method for LOESS-based seasonal trend decomposition that was proposed by (Robert B Cleveland et al., 1990). LOESS first fits polynomials to each subset of the data to obtain a weighted regression curve, respectively, and then joins the centers of these regression curves together to form a complete regression curve (Cleveland and Devlin, 1988). SLT is able to separate the time series trend into three components: seasonal component, trend-cycle component, and remainder component. If there are outliers in the time series data, they will be reflected in the residual.

Isolated Forest:
The Isolated Forest is an anomaly detection method based on a probability model (Liu et al., 2008). For applying an isolated forest to a time series, the data set is recursively and randomly partitioned until all sample points are isolated. Normal data distribution with high density is required to be segmented many times to be isolated, but those anomalous data points with very low density can be isolated easily. With this random segmentation strategy, the anomalies usually have short paths. The short path indicates a few numbers of segments.

ARIMA: ARIMA (Auto regressive Integrated Moving
Average model) is a classical prediction-based approach (Moayedi and Masnadi-Shirazi, 2008;Zhang, 2003) that has been widely used for time series forecasting and anomaly detection. In ARMA, the first part of the time series is set as historical data to train the model, and the historical data is then used to predict future data. The anomaly points are obtained by comparing the difference between the true value and the predicted value.

Breakout detection:
The Breakout detection is one of the advanced statistical unsupervised anomaly detection algorithms (James et al., 2016). The basic principle of the Breakout detection model is to use energy statistics to detect the divergence of averages, referred to as E-Divisive with Medians (EDM).

Autoencoder:
Autoencoder algorithm is an unsupervised anomaly detection method based on deep learning (Sakurada and Yairi, 2014). Autoencoder is essentially a dimensionality reduction method based on neural networks for high-dimensional inputs with low-dimensional outputs. It contains two parts, which are encoder and decoder. The encoder is used to learn a low-dimensional representation of the input data, while the decoder reproduces the input data in the original dimension using the low-dimensional representation generated by the encoder, also called reconstruction. After Autoencoder is trained with a large amount of normal data, it can reproduce the input data well when fed with data "similar" to the data which were used for training. However, if the input data contain outliers, the trained Autoencoder model is unable to reproduce the input data correctly.

Time series anomaly detection strategy combining Breakout detection and LSTM
The coherence time series, affected by the surface vegetation shows seasonality, with smaller coherence values in the season of dense vegetation and larger values in the season of sparse vegetation. Vegetation growth is closely related to annual rainfall, but the rainy seasons are not perfectly the same from year to year. Therefore, although the time series of coherence data roughly show seasonality, the coherence values are not exactly similar in the same months of each year. This is a challenging task for Breakout detection, K-means, and STL anomaly detection methods as the inconsistent seasonality will interfere with the anomaly detection, and eventually lead to errors in the anomaly detection results. Although anomaly detection methods based on prediction model, probabilistic model, and deep learning are unsupervised techniques, they require a set percentage of hypothetical anomaly to the total data. However, if the actual training samples have a high percentage of anomalous samples, which violate the basic assumption of anomaly detection, the true false alarm rate achieved during anomaly detection may be much greater than expected. Therefore, for anomaly detection in time series of coherence, we cannot set a perfect percentage of anomalies. If only Sentinel-1 data are used, the outliers can be even as high as 50% when the landslide occurs earlier than 2017. The imprecision of the initial outlier percentage setting makes the detection of outliers seriously deviate from the true outliers. Moreover, not enough training data maybe available. For the landslide occurrence time detection using Sentinel-1 data the earliest available data starts only in 2014. For example, for the Shaziba landslide in this study, only about 170 images were available until July 2021. Unsupervised techniques based on deep learning are particularly difficult when the training data are small (Srivastava et al., 2014).
To address the above challenges, we propose a new strategy that combines the unsupervised learning method with the supervised learning method based on deep learning to achieve a more accurate automatic detection of landslide occurrence time without a priori knowledge. We use an advanced deep learning LSTM model, which can predict the time series in the future from its historical time series in the past, and then determine whether the future sequence is anomalous based on the prediction error (Ergen, 2020;Hochreiter and Schmidhuber, 1997;Malhotra et al., 2016). The LSTM model has three main phases: forgetting phase, selective memory phase, and output phase. The forgetting stage is mainly to selectively forget the input passed in from the previous node. In simple terms, it means that the unimportant information is forgotten and the important information is remembered. The selective phase indicates that there is a choice to remember the important input information. The output phase determines which part of important information will be taken as the output. The LSTM neural network architecture is shown in Figure 1. To apply LSTM, firstly, historical data are needed to be labelled as training and validation set. The model is then trained using the training data. Finally, the trained LSTM model can predict the future data. However, without any prior knowledge about the failure time, we cannot label the time series data properly. To address this problem, we use the Breakout detection to assist in labelling the training data. Breakout detection has advantages in seasonal time series anomaly detection to highlight a level shift in time series. The level shift refers to the increase or decrease of the value between minimum period, which is set as 1 year for our study. By applying the Breakout detection on coherence time series, we can detect the year in which the input has outliers. The coherence data before the occurrence of anomalies are then labelled as normal data, which we use as input to the LSTM model for training. The trained LSTM model is then used to predict the subsequent coherence data. When the difference between the true coherence values and the predicted ones is greater than a certain threshold, an abnormality occurs. In this paper, we made an experiment using July 21, 2020 Shaziba landslide in China and compared the performance of our proposed strategy for failure detection time with widely used unsupervised algorithms. The Shaziba landslide blocked the Qingjiang River at the foot of the slope forming a weir that threatened nearby villages and the city of Enshi downstream (Fig. 2). More information about this landslide can be found in the following references (Shen et al., 2021;Song et al., 2021;Xue et al., 2022).

landslide occurrence time from optical and SAR data
Only 41 cloud-free Sentinel-2 multi-temporal data were available for the landslide area from January 1, 2016, to February 28, 2021. Figure 2 shows the time-series of NDVI (blue line) covering the landslide zone. After the event, the NDVI time series remains at its minimum level. This contrasts with other minima occurring before the landslide failure in the NDVI time series, where each minimum is followed by a sudden increase afterwards. This change in temporal behavior can be used to detect the timing of the landslide failure. Using our prior knowledge of the exact timing of the landslide failure, we found that the NDVI values closest to the failure before and after the event were on May 30, 2020, and August 4, 2020, respectively. In this case, we are in a fortunate situation that a cloud-free data take was available not long after the event. This may not be the case for the other regions. Therefore, based on the analysis of the NDVI values we can determine the time period of the landslide failure between May 30, 2020, and August 4, 2020.  However, in case the ground surface is characterized by high coherence before the landslide failure, or the vegetation regrows rapidly after the failure, it will be more challenging to use coherence time-series to detect the timing of the failure. This necessitates development of a data-driven deep learning method for automatically detecting the failure time.

Automatic detection of failure time
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France      Figure 7 shows the anomalies detected based on this approach. The SLT detects anomalies based on residuals. With a threshold confidence interval of 95%, we used the mean of the deviation of the residuals as the threshold to determine anomalies in the time series. As seen in Fig. 7, with the exception of July 11, 2020, which is closest to the real failure time, other anomalies detected on June 17, 2020 to July 21, 2022 are obviously not correct. As STL is suitable for seasonal time series, it can detect almost all anomalies in seasonal time series. However, the earliest time detected is June 17, 2020, which is even one month earlier than the true occurrence of the Shaziba failure on July 21, 2020.

Figure 8.
Anomaly detection result based on the K-means method. Red dots represent anomalies. Blue dots represent normal. Figure 8 shows the anomalies detected based on the K-means method. As seen in Fig. 8, the result fails to correctly detect the real failure time. K-means anomaly detection is based on clustering normal data instead of clustering anomalous data. In addition to its statistical properties, time series data also have time-dependent trend and seasonality. As the K-means method does not take into account these components and only treats coherence time series as common data sets without any seasonality and trends, it leads to false detection of the failure time. Figure 9. Anomaly detection result based on the Isolate Forest method. Red dots represent anomalies. Blue dots represent normal. Figure 9 shows the anomaly detection result based on the Isolate Forest method. The results show anomalies on January 4, 2017, January 18, 2019, July 11, 2020, March 20, 2021, April 1, 2021, and April 25, 2021. Therefore, the failure occurrence time cannot be detected correctly. Isolated Forest is an anomaly detection method based on decision trees. The strategy of Isolated Forest is to identify anomalies instead of analyzing normal data points. The method determines whether a data point is anomalous by predicting the probability that it may be anomalous for each data point. But it needs to assume the proportion of abnormal values in advance, which may not be fulfilled in the coherence time series. When the percentage of outliers differs from the assumption within a reasonable range, the abnormal data can be detected perfectly. However, when the contamination of outliers is much higher than the assumed value, there will be a large number of false alarms and missed judgments. In this study, the anomalous coherence accounts for roughly 27% of the contamination level. This is the reason why the detection is not accurate enough.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France Figure 10. Anomaly detection result based on the ARIMA forecasting method. Red dots represent anomalies. Blue dots represent normal. Figure 10 shows the anomaly detection result based on the ARIMA method. It detects potential failures on July 11, 2020, August 16, 2020, August 28, 2020, and September 10, 2020. The first detected anomaly is 10 days earlier than the true time, while the others after the event. As described in the method, the ARIMA model works based on the prediction strategy that predicts future point data from the historical data. Anomalies are identified if the difference between the predicted data and the real data is greater than the difference threshold. However, the proportion of historical data needs to be set in advance. We used the default value of 0.8 in this study. This is the same challenge as for the isolated forest model. Therefore, historical data scale setting error is the reason for false positives in ARIMA anomaly detection. Figure 11. Anomaly detection result based on the Breakout detection method. Red dots represent anomalies. Blue dots represent normal. Figure 11 shows the time detection result based on the Breakout detection method. The results suggest that the failure occurred in the landslide area one month earlier on June 17, 2020. The Breakout detection method can detect sudden jumps in a time series, or gradual increases/decreases from one steady state to another. The method has a periodicity constraint parameter (min_size) that is particularly friendly to periodic and seasonal time series. This parameter indicates the minimum period of the time series. Once a Breakout is detected in a given period, another Breakout cannot be detected in this period. The min_size was set to 31 in this study, as we selected 31 timeseries Sentinel-1 images per year for the analysis of the Shaziba landslide. Figure 12. Anomaly detection result based on the Autoencoders method. Red dots represent anomalies. Blue dots represent normal. Figure 12 shows the anomaly detection result using the Autoencoders method. The algorithm identifies the failure time on June, 2021, which is not in agreement with the reality. Although Autoencoder algorithm is an advanced deep learning method, the method has led to the worst result compared to other methods. The main reason could be due to lack of enough training data and incorrect labelling of them, which increase the chance for false alarms. Although the Breakout detection method cannot accurately detect real outliers related to failure time, it can be used to assess if each minimum period sustains any outliers. Therefore, the anomaly detection result using Breakout detection method as shown in Figure 11 can be used as pre-analysis step to determine that the failure occurred in 2020 and not in 2016-2019 and 2021. Therefore, we labelled the time series of between coherence 2016 and 2019 as normal data and used them as the training data for the LSTM model. Then, the trained LSTM model and the time series coherence of 2019 are used to predict the time series coherence of 2020 and 2021. As shown in Figure 13, the blue curve represents the real coherence time series and the yellow curve represents the predicted coherence time series. From the result, we observe that the predicted coherence time series and the real coherence time series match each other very well before the failure, but the correlation between the two decreases dramatically after the failure. We use the mean of the standard deviation and the difference between the predicted and true values as the anomaly detection threshold. Anomalies are indicated when the standard deviation and difference values are greater than the threshold value. This strategy identifies the failure time on 23 July, 2021, an estimation that is more accurate than other methods. Therefore, we can automatically and accurately detect the time of failure occurrence from the coherence time series by combining unsupervised learning of Breakout detection and supervised learning of LSTM prediction models without a priori knowledge of whether a landslide failure has occurred.

CONCLUSION
In this paper, we apply a deep learning strategy to SAR coherence time series for automatically detecting the time of landslide occurrence. Our strategy is a combination of Breakout detection algorithm for labelling data, and LSTM model for predicting data. Experimental results using coherence timeseries derived from Sentinel-1 satellite for the July 21, 2020 Shaziba landslide show that compared to commonly-used unsupervised anomaly detection methods, the use of our strategy leads to a more accurate estimation of the landslide failure time. In the future, we will include more case studies into our experiments to better evaluate the advantages and shortcomings of our methods for landslide failure detection time for different types of landslides and various geographic locations.