RELATIVE ERROR EVALUATION TO TYPICAL OPEN GLOBAL DEM DATASETS IN SHANXI PLATEAU OF CHINA

Produced by radar data or stereo remote sensing image pairs, global DEM datasets are one of the most important types for DEM data. Relative error relates to surface quality created by DEM data, so it relates to geomorphology and hydrologic applications using DEM data. Taking Shanxi Plateau of China as the study area, this research evaluated the relative error to typical open global DEM datasets including Shuttle Radar Terrain Mission (SRTM) data with 1 arc second resolution (SRTM1), SRTM data with 3 arc second resolution (SRTM3), ASTER global DEM data in the second version (GDEM-v2) and ALOS world 3D-30m (AW3D) data. Through process and selection, more than 300,000 ICESat/GLA14 points were used as the GCP data, and the vertical error was computed and compared among four typical global DEM datasets. Then, more than 2,600,000 ICESat/GLA14 point pairs were acquired using the distance threshold between 100m and 500m. Meanwhile, the horizontal distance between every point pair was computed, so the relative error was achieved using slope values based on vertical error difference and the horizontal distance of the point pairs. Finally, false slope ratio (FSR) index was computed through analyzing the difference between DEM and ICESat/GLA14 values for every point pair. Both relative error and FSR index were categorically compared for the four DEM datasets under different slope classes. Research results show: Overall, AW3D has the lowest relative error values in mean error, mean absolute error, root mean square error and standard deviation error; then the SRTM1 data, its values are a little higher than AW3D data; the SRTM3 and GDEM-v2 data have the highest relative error values, and the values for the two datasets are similar. Considering different slope conditions, all the four DEM data have better performance in flat areas but worse performance in sloping regions; AW3D has the best performance in all the slope classes, a litter better than SRTM1; with slope increasing, the relative error for the SRTM3 data increases faster than other DEM datasets; so SRTM3 is better than GDEM-v2 in flat regions but worse in sloping regions. As to FSR value, AW3D has the lowest value, 4.37%; then SRTM1 data, 5.80%, similar to AW3D data; SRTM3 has higher value, about 8.27%; GDEM-v2 data has the highest FSR value, about 12.15%. FSR can represent the performance of correctly creating the earth surface based on DEM data. Hence, AW3D has the best performance, which is approximate to but a little better than SRTM1. The performance of SRTM3 and GDEM-v2 is similar, which is much worse than AW3D and SRTM1, and the performance of GDEM-v2 is the worst of all. Originated from the DEM dataset with 5m resolution, AW3D is regarded as the most precise global DEM datasets up to now, so it may exerts more effect in topographic analysis and geographic research. Through analysis and comparison of the relative error for the four open global DEM datasets, this research will provide reference in open global DEM datasets selection and applications in geosciences and other relevant fields. * Corresponding author


INTRODUCTION
As the data source in digital topographic analysis, digital elevation model (DEM) data plays an important role in geoscience researches (Jarihani et al., 2015;Moore et al., 2011).Mainly produced by radar data or stereo remote sensing image pairs, global DEM datasets are one of the most important types for DEM data (O'Loughlin et al., 2016).The successive release of open global DEM datasets provides multi choices for topographic research and applications (Suwandana et al., 2012;Patel et al., 2016).Meanwhile, the feasibility of the applications using DEM data depends on its accuracy (Dragut and Eisank, 2011;Mukherjee et al., 2013).Hence, accuracy assessment to the open global DEM datasets has much significance, which is also an import part of quality evaluation to remote sensing data (Berry et al., 2007;Nikolakopoulos et al., 2006;Zhao et al., 2011).Traditional accuracy assessment to DEM data mainly adopts vertical error based on large accurate ground control points (GCPs), that is, the difference of the elevation values between GCPs and DEMs (Avtar et al., 2015;Hirano et al., 2003).Vertical error can be used to correct the values and improve the quality of the DEM data (Yue et al., 2010;Zhao et al., 2015).However, vertical error mainly focuses on elevation accuracy and seldom considers the errors of the neighbouring pixels, which is called as relative error (Satge et al., 2016).Relative error relates to surface quality created by DEM data (Leigh et al., 2009), so it relates to geomorphology and hydrologic applications using DEM data (Bamber, 1994;Siart et al., 2009).

Study Area
This research takes Shanxi Plateau of China as the study area.Located in middle and northern part of China, Shanxi Plateau mainly distributes in Shanxi Province (Figure 1).The study area is approximate rhombus, narrow in east-western direction and long in north-southern direction.The middle part mainly distributes basins, high in northern section but low in southern section.Western and eastern parts are mountainous regions, which are Lvliang Mountains and Taihang Mountains respectively.SRTM1 and SRTM3 data were collected in February 2000, which cover the land between 60ºN and 56ºS, about 80% of global land.SRTM3 data was firstly released by the United States Geological Survey in 2003, which achieved great successes due to homogeneous-quality surface and high spatial resolution.In 2015, SRTM1 was freely released by NASA's Earth Observing System Data and Information System website (http://reverb.echo.nasa.gov/).There were some little holes in SRTM1 data.Fortunately, there were no holes for SRTM1 data in the study area (Fig. 1).So it is significant to evaluate the quality and accuracy among SRTM1 and other DEM datasets in Shanxi Plateau.
Both acquired by global ASTER stereo images, GDEM-v2 is an upgraded version of GDEM-v1 data.It is developed using an advanced algorithm and more data sources (Zhao et al., 2015).
With higher spatial resolution (1″, about 30m) and wider coverage (83ºN-83ºS) than SRTM3, GDEM-v2 data acquires enough attentions after its release in October 2011 from USGS Global Data Explorer.
With height accuracy of 5 m, AW3D data released in March 2017 covers global land between 60ºN and 60ºS, which is approximate to SRTM data.Meanwhile, all the holes are filled by existing DEM datasets.The grid value in AW3D is calculated by average or median when resampling from the "World 3D Topographic Data" with 5m spatial resolution, so AW3D data is regarded as the most precise global DEM dataset at 30m resolution level.The "World 3D Topographic Data" with 5m spatial resolution may be freely released in the future, so it is much significant to estimate the performance of AW3D data in advance.
ICESat/GLA14 data can be downloaded from the U.S. National Snow and Ice Data Centre (NSIDC), which is collected from January 2003 to February 2010 (Zwally et al., 2002).The ICESat/GLA14 data is point data with footprint size of about 70 m and the distance of about 172 m between two points.The inter-track spacing distance is 30 km near the equator.Zwally et al. (2002) showed that the vertical accuracy is approximately 15 cm at the global scale.So the accuracy of the ICESat/GLA14 data is much higher than that of the four open global DEM datasets.It is reasonable to take ICESat/GLA14 points as the reference to assess the performance of these DEM datasets.

METHODOLOGY
Firstly, ICESat/GLA14 data was processed as a reference; then, Point Pairs were constructed and relative error was computed accordingly for the four DEM datasets; finally, FSR index was acquired and all the results were analyzed in different slope classes.

ICESat/GLA14 Data Processing
ICESat/GLA14 data were firstly extracted by using NGAT tools downloaded from NSIDC, and then transform them into point files.Overlapping the ICESat/GLA14 point data at all the phases in Shanxi Plateau, the original ICESat/GLA14 data was acquired.
Then, the ellipsoid transformation was conducted between TOPEX/Poseidon and WGS 84 ellipsoids to make consistency.
Finally, taking 50m as the threshold, the difference between the ICESat/GLA14 data and the four DEM datasets were computed, and all the points higher than the threshold were removed.Through this process, the total number of the ICESat/GLA14 points decreased from 331, 817 to 316,148, which were the processed ICESat/GLA14 data in this research.

Point Pairs Construction
Point pair was the basis to compute the relative error and FSR index.Considering the spatial resolution of the SRTM3 data was 90m, the minimum threshold of the distance between the two points for the point pairs was fixed at 100m.Through analysis, the maximum threshold was determined at 500m.Then, the points were removed from the point pairs when their elevation difference between the DEM datasets and the ICESat/GLA14 data was lower than 1 m.After process, the total number of 2,655,382 point pairs was acquired to compute relative error and FSR index.

Relative Error Calculation
Relative error was computed for all the point pairs, the equation of which is the following: (2) where n = the total number of the points x = the value of the DEM datasets y = the value of the ICESat/GLA14 points

FSR Calculation
The slope trend for every point pair can be shown in Figure 2: Figure 2 shows slope trend is consistency for DEMs and GCPs (ICESat/GLA14 points) in case "A", which is inconsistency in case "B".
FSR index is to summarize the occurrence percentage of the case "B", the equation of which is the following (Satge et al., 2016): where FSR = false slope ration A = point pair in case "A" B = point pair in case "B"

Slope Class Distribution
Relative error and FSR index were not only analyzed in the whole study area, but in all the slope classes.Based on SRTM1 data, slope index was computed using the topographic analysis tools in ArcGIS software.Then, the slope index was divided into five classes according to the values, which was in Figure 3: Figure 3 shows: low slope distributes in the middle plains; high slope mainly distributes in margin mountainous regions, especially in western mountains.Moreover, the area in every slope class was approximate, which guaranteed the analysis results in slope class distribution.

RESULTS
Relative error was acquired for the four measures for the four DEM datasets, which was also analyzed in different slope classes; then, FSR index was calculated in the whole study area and different slope classes.

Relative error of typical DEM datasets
Four measures of the relative error for four typical open global DEM datasets were calculated as in Table 2 shows: AW3D is better than SRTM1, which have higher accuracy.SRTM3 is better than GDEM-v2 in flat regions but worse in steep regions.

FSR of typical DEM datasets
As the total number of the point pairs is known, case "A" can be calculated when the case "B" is given, so case "A" is omitted in the computation.Through calculating the number of case "B" for the point pairs, FSR index for the four DEM datasets are in Table 3:  3 shows: AW3D is the lowest, a little lower than SRTM1; the next is SRTM3; GDEM-V2 data has the highest FSR value, which is much higher than other three DEM datasets.

FSR of typical DEM datasets based on different slope classes
To acquire B and FSR values for different slope classes for the four typical global DEM datasets, the calculation results are in  et al., 2007;Nikolakopoulos et al., 2006;Zhao et al., 2011), this research evaluate the relative error of the DEM datasets through point pairs using ICESat/GLA14 data.
Even compared to the FSR evaluation research by Satge et al. (2016), this research adopts more global DEM datasets types and using relative error index which consider the distance effect.
5.2 Prospects DEM is mainly used to calculate all kinds of topographic indexes in geosciences researches.The quality and feasibility of these indexes can be evaluated in the future, which may represent the accuracy of the DEM data indirectly.

CONCLUSIONS
(1) AW3D has the best performance, which is approximate to but a little better than SRTM1.The performance of SRTM3 and GDEM-v2 is similar, which is much worse than AW3D and SRTM1, and the performance of GDEM-v2 is the worst of all.Originated from the DEM dataset with 5m resolution, AW3D is regarded as the most precise global DEM datasets up to now, so it may exerts more effect in topographic analysis and geographic research.
(2) About relative error, AW3D has the lowest values in mean value, mean absolute value, root mean square value and standard deviation value; then the SRTM1 data, its values are a little higher than AW3D data; the SRTM3 and GDEM-v2 data have the highest relative error values, and the values for the two datasets are similar.
(3) Considering relative error in different slope conditions, all the four DEM data have better performance in flat areas but worse performance in sloping regions; AW3D has the best performance in all the slope classes, a litter better than SRTM1; with slope increasing, the relative error for the SRTM3 data increases faster than other DEM datasets; so SRTM3 is better than GDEM-v2 in flat regions but worse in sloping regions.
(4) As to FSR value, AW3D has the lowest value, 4.37%; then SRTM1 data, 5.80%, similar to AW3D data; SRTM3 has higher value, about 8.27%; GDEM-v2 data has the highest FSR value, about 12.15%.FSR can represent the performance of correctly creating the earth surface based on DEM data, which has the same indication with the relative error.
(5) Analysing the FSR value in different slope classes, AW3D has the lowest value in all the classes; SRTM3 has similar value in all the classes; as to other three DEM datasets, AW3D and SRTM1 has the lowest values in the middle slope classes, while GDEM-v2 continuously decreases with slope increasing.
China as the study area, this research aims to evaluate the relative error of typical open global DEM datasets including Shuttle Radar Terrain Mission (SRTM) data with 1 arc second resolution (SRTM1), SRTM data with 3 arc second resolution (SRTM3), ASTER global DEM data in the second version (GDEM-v2) and ALOS world 3D-30m (AW3D) data based on the ICESat/GLA14 points.Meanwhile, false slope ratio (FSR) index was computed through analyzing the difference between typical global DEM datasets and ICESat/GLA14 values.Both relative error and FSR index were categorically compared for the four DEM datasets under different slope classes.Through analysis and comparison of the relative error for the four open global DEM datasets, this research will provide reference in open global DEM datasets selection and applications in geosciences and other relevant fields, especially in the study area, Shanxi Plateau of China.

Figure 1 .
Figure 1.Study area and data sources 1) where RE = relative error ∆H = elevation difference D = horizontal distance Relative error was analyzed in different measures, including mean error (ME), mean absolute error (MAE), root mean square error (RMSE) and standard deviation error (STD).The equations of these measures were the following:

Figure 3 .
Figure 3. Slope class distribution in Shanxi Plateau

Table 1 .
Relative Error of typical DEM datasets (m)Table1shows: ME is about 0m four every DEM dataset, so RMSE and STD are the same, and only RMSE will leave when evaluate the relative error in different slope classes.As to MAE and RMSE, AW3D is the lowest, then the SRTM1; SRTM3 and GDEM-v2 are approximate, which are much higher than AW3D and SRTM1.

Table 2 .
Relative Error of typical DEM datasets based on different slope classes (