UNCERTAINTY ASSESSMENT AND WEIGHT MAP GENERATION FOR EFFICIENT FUSION OF TANDEM-X AND CARTOSAT-1 DEMS

Recently, with InSAR data provided by the German TanDEM-X mission, a new global, high-resolution Digital Elevation Model (DEM) has been produced by the German Aerospace Center (DLR) with unprecedented height accuracy. However, due to SAR-inherent sensor specifics, its quality decreases over urban areas, making additional improvement necessary. On the other hand, DEMs derived from optical remote sensing imagery, such as Cartosat-1 data, have an apparently greater resolution in urban areas, making their fusion with TanDEM-X elevation data a promising perspective. The objective of this paper is two-fold: First, the height accuracies of TanDEM-X and Cartosat-1 elevation data over different land types are empirically evaluated in order to analyze the potential of TanDEM-XCartosat-1 DEM data fusion. After the quality assessment, urban DEM fusion using weighted averaging is investigated. In this experiment, both weight maps derived from the height error maps delivered with the DEM data, as well as more sophisticated weight maps predicted by a procedure based on artificial neural networks (ANNs) are compared. The ANN framework employs several features that can describe the height residual performance to predict the weights used in the subsequent fusion step. The results demonstrate that especially the ANN-based framework is able to improve the quality of the final DEM through data fusion.


INTRODUCTION
Digital Elevation Models (DEMs) as a representation of terrain surface with different resolution and coverage are produced for diverse applications.Special interest lies in the generation of global DEMs, which provide homogeneous information about the worlds topography for (almost) the complete globe.One of the most recent and most prominent examples of global terrain datasets is the TanDEM-X DEM, which was produced by the German TanDEM-X mission, consisting of two SAR satellites flying in close orbit configuration.The mission goals aimed at a DEM of HRTI 3 standard, i.e. with a relative height accuracy better than 2 m for slopes lower than 20 % (Krieger et al., 2007).However, initial visual inspection of TanDEM-X DEM data demonstrates undesirable horizontal resolution and height accuracy in morphologically complex terrains like urban areas and in consequence the need for quality enhancement in these areas (Rossi et al., 2011).One possibility for improvement of TanDEM-X elevation data in problematic terrain is the fusion with other available elevation data which do not suffer from SAR-inherent imaging effects such as layover and shadowing.Examples for these alternative data are DEMs derived from optical stereo imagery.Optical DEMs generally reach HRTI-3 standard (with a relative height accuracy of 2-3 m) when they are produced using high-resolution optical imagery like acquired by Cartosat-1.This paper first investigates the height accuracies and errors of both the TanDEM-X and the Cartosat-1 elevation data for different land types.The output of this assessment illustrates the performance of each DEM for a specific land type and allows for an educated judgment about which DEM shows favourable accuracy for which land type, thus providing a basis for the consideration of DEM fusion.Afterwards, the results of a simple preliminary TSX/Cartosat-1 DEM fusion are shown and discussed.In order to reach a better result, a more sophisticated approach for adaptive weight map generation is proposed.It employs several features that can describe the height residual performance and uses them as input to an artificial neural network (ANN) to predict the weights used in the subsequent fusion step.

TANDEM-X AND CARTOSAT-1 DEM UNCERTAINTY ASSESSMENT
Because of the different vertical and horizontal datum of the test DEMs and the reference DEM, all data must first be transferred to one reference datum with identical pixel spacing.For the experiments in this paper, we chose to use UTM as joint reference system with heights measured above the WGS84 ellipsoid.All data were resampled to 3 m pixel spacing to preserve more details in the high resolution DEM and to prevent the creation of an artificial vertical bias caused by lost building shape details.
After datum homogenization, the test DEMs must be precisely aligned to the reference DEM in order to remove any rotational and horizontal translations.In this study, the Iterative Closest Point (ICP) algorithm (Ravanbakhsh and Fraser, 2013) was used for DEM coregistration.After alignment, the DEMs can be evaluated with respect to the reference dataset.The usual metrics for this aim are standard deviation (STD) and LE90 (linear error in 90% confidence interval; a common metric in the TanDEM-X specification document to express TanDEM-X accuracy locally and globally (German Aerosapce Center (DLR), 2016).All aforementioned metrics are only reasonable when the height errors at least approximately follow a normal distribution.In addition, If the quantile-quantile plot of the height errors over study areas demonstrates that the errors do not follow a normal distribution, other robust metrics like normal median absolute deviation (NMAD) are recommended for error analysis (Höhle and Höhle, 2009).Section 4.2 summarizes the results of our accuracy assessment of TanDEM-X and Cartosat-1 DEM data over different urban and rural land types.

DEM FUSION OVER URBAN AREAS
As the results of the uncertainty assessment (see Section 4.2) will validate, the quality of the TanDEM-X DEM is not perfect in urban areas, especially when high buildings are present.As mentioned in Section 1, a feasible solution for the improvement of TanDEM-X elevation data over urban areas is the fusion with Cartosat-1 DEM data, which were shown to provide much better accuracy in urban areas.Various methods have been formulated for DEM fusion (Crosetto et al. 1998;Reinartz et al. 2005;Papasaika et al. 2011;Kuschk et al. 2016;Fuss et al. 2016), an important application case of data fusion in the field of remote sensing (Schmitt and Zhu, 2016).Among the aforementioned methods, the simplest and most popular one is weighted averaging (WA) (Reinartz et al., 2005).The main issue of WA is to employ weights which are proportional to the expected height errors.In this paper two attempts are followed for generating weight maps which can describe the qualities of corresponding DEMs.First, the height error maps delivered with the TanDEM X data and the standard deviation maps of the Cartosat 1 stereo matching are used as weight maps in the WA-based fusion.Secondly, more sophisticated weight maps are generated using an ANN-based approach in order to predict more accurate weights based on spatial features extracted from the DEMs.

DEM Fusion Using HEM-based Weight Maps
The TanDEM-X DEM comes with a height error map (HEM), which is derived from the coherence estimates, the number of looks, and the baseline configuration.For the Cartosat-1 DEM, the standard deviation of the stereo matching process, including consideration of the number of stereo matched points can be used to produce a similar HEM.
Before application of the fusion process, the Cartosat-1 DEM must be aligned to the TanDEM-X DEM, which again is carried out using the ICP algorithm.After that, TanDEM-X and Cartosat-1 are fused together by using the acquired weight maps based on the simple weighted averaging.

DEM Fusion Using ANN-Predicted Weight Maps
As will be shown in Section 4.3.1,DEM fusion over urban areas using simple weighted averaging with weight maps derived from the height error maps delivered with the DEM products does not result in a DEM accuracy exceeding the Cartosat-1 DEM accuracy.Therefore, a more sophisticated approach for adaptive weight map generation is proposed.It employs several features that can describe the height residual performance and uses them as input to an artificial neural network (ANN) to predict the weights used in the subsequent fusion step.The framework of the proposed method is shown in Fig. 1.It consists of three main steps that are: 1) feature extraction and height residual computation, 2) data preprocessing and finally 3) training of an artificial neural network to generate a suitable weight predictor.The output of the network is a model that can approximately predict the height residuals in the target areas where two DEMs are supposed to be fused.

Feature Extraction and Data Preprocessing:
For the training of the ANN training data representing different relevant land types are used.From those, features such as slope, aspect, edginess, different types of roughness, anisotropic coefficient of variation (ACV) and height error map (HEM) are extracted (Olaya, Gruber and Peckham, 2009).In addition, height residual maps are calculated from the corresponding DEM patches and LiDAR ground truth data.
Before building the ANN structure, pre-processing the height residuals related to feature values is essential to remove outliers and reduce the noise influence.For this purpose, a two-step mean filtering process is carried out.The significant characteristic of this pre-processing step is to bin feature values by a statisticalempirical binning approach.The feature values and their corresponding height residuals are binned by Freedman-Diaconis rule (Birgé and Rozenholc, 2006): where h = 2 × IQR × n −1/3 , and N is the number of bins, f max j and f min j are the max and min values of feature j, and h is the bin width.IQR is the interquartile range and n is the number of measurements.
The mean filter is bin-wisely performed for each bin to smooth height residual values.This process should be carried out for each feature.Then corresponding to each pixel, the new outcomes of the previous step as smoothed height residual are averaged again to give a height residual for every pixel.After pre-processing, the data used to train the ANN for weight map prediction.

ANN for Weight Map Prediction:
The filtered inputs from Section 3.2.1 are then used to train a classical artificial neural network.The final output of the trained ANN is a predictive weight map model that can be used for arbitrary parts of DEMs in order to anticipate the height residuals just based on measured features.The explained framework is implemented for both kinds of DEMs to generate an individual weight map for each DEM separately.The predicted residual maps can then be used as weight maps associated with each kind of DEMs in the fusion process.
For training the NN, training data representing versatile land types, e.g.inner city area (densely packed, relatively high buildings), residential areas (single family homes and detached buildings), agricultural areas, and forested areas are used in order to ensure generalizability.After feature extraction, height residual computation and preprocessing based on the pipeline described in Section 3.2.1, the output is input into to the NN.The NN is trained using the filtered feature vectors as inputs and the modified height residuals as outputs.70% of the data are allocated for training, 15% for validation, while the rest is devoted as testing data to monitor the performance of the network during the training.

Study Area and DEMs
Cartosat-1 is an Indian satellite equipped with a dual-optics 2line along-track stereoscopic push broom scanner with a stereo For height accuracy evaluation, six subsets of different land types are selected: 1. Industrial area (Sub A).This land type is characterized by flat, simple and rectangular buildings which often are not very high.2. Inner city area (Sub B).This land type includes many dense buildings and possibly also some higher buildings.3. High buildings (Sub C) are skyscrapers and other high rise buildings.4. Residential area (Sub D).This type contains many small one-family homes 5. Forested and agricultural area (Sub E). 6. Agricultural areas and lake (Sub F).
Figure 2 shows the locations and descriptions of these study subsets.All these test areas are located in the area of Munich, Bavaria, so that for each of them a highly-accurate LiDAR point cloud (with a density of 1 point per m) provided by the Bavarian surveying administration is available as reference.

Accuracy Assessment Results
Figure 3 shows the vertical offset of the TanDEM-X and the Cartosat-1 DEM in comparison to the reference DEM in each subset.The vertical bias of Cartosat-1 depends on the absolute vertical accuracy of the external DEM that is used for RPC refinement because the Cartosat-1 RPCs originally only provide an accuracy in the order of several hundred meters.On average, the vertical offset of the Cartosat-1 elevation data over the different test areas is 1.327 m.The vertical offset of the TanDEM-X DEM depends on the accuracy of determining phase offset.For this task, the initial radargrametric DEM of coarse accuracy is produced by TanDEM-X bistatic images and orbital information.Then, this DEM is employed as an external DEM in phase unwrapping process.The phase offset is calculated from the mean of the most probable values in the phase difference histogram (histogram of radargrammetric and unwrapped phase difference), whereas layover and shadow areas are disregarded (Rossi et al., 2012).In consequence, for areas affected by layover or shadowing, such as subset C (high buildings), the vertical offset is higher than in the other test subsets because they are discarded for phase offset estimation and the vertical offset estimate is not valid for these areas.The mean height offset of the TanDEM-X DEM over all subsets is found to be -0.794m.In contrast, the height offset of the TanDEM-X DEM calibrated with ICESat data is even lower (Gruber et al., 2012).
The uncertainty assessment results over urban and non-urban areas after vertical bias removal are collected in Tab. 1.The height residual maps for TanDEM-X and Cartosat-1 DEM in study areas are also displayed in Figs.4-9.It has to be noted that all figures and mean heights are derived by comparing the DEMs and the LiDAR reference data after vertical bias correction.The main problem of using the matching standard deviation as HEM for optical DEMs like Cartosat-1 is that it is not computed comprehensively for urban subsets and thus is only available in a rather sparse distribution.Thus, the Cartosat-1 HEM was pixelwisely calculated based on the mean standard deviation of the available values derived from the dense matching step proportional to the number of pair-rays in each cell in urban areas.Unfortunately, the TanDEM-X HEM does not consider the layover effect, and as a result the mean of TanDEM-X HEM is lower than mean of the Cartosat-1 HEM, while, using prior knowledge of DEM qualities from the previous uncertainty assessment, it is known that Cartosat-1 is more accurate than the TanDEM-X DEM for urban areas on average.The assessments reveal the variance of TanDEM-X HEM is higher than Cartosat 1 HEM.Thus, the 3σ of HEM values can be used as appropriate ratio for normalized weight map generation from the HEMs.The fusion was just carried out in urban areas in subset A, B, C, and D because the standard deviation of matching was not available for non-urban areas.The results of fusion are summarized in Tab. 2. The performance of the full DEM fusion chain was then evaluated on a completely independent subset as target area (Sub A) with the size of 238 x 298 = 70924 pixels.The TanDEM-X and Cartosat-1 DEMs of this subset are fused together using a pixel-  wise weighted averaging algorithm using the obtained height error maps as weights of the input DEMs.In Table 3, the results of the fusion by adopting the proposed framework for weight map generation based on the neural network are compared with other methods like HEMs as weights and also just using the simple averaging without using weights.

DISCUSSION
The results of accuracy assessment of TanDEM-X and Cartosat-1 DEM collected in Table 1 show that the Cartosat-1 DEM has a higher height accuracy than the TanDEM-X DEM in urban areas.
The difference of two DEMs is lowest in residential areas and maximal in the high building subset.The main source of errors in the TanDEM-X DEM come from the layover effect that causes a phase ramp (sometimes referred to as front porch effect (Thiele et al. 2007)), which leads to wrong height reconstructions.Since the amount of layover depends on the height of the buildings and the incidence angle of SAR sensor, its effect is less for lower residential and industrial buildings and increases for inner city areas with more high buildings, while being maximal for areas with skyscrapers.The layover causes the height of the left side of building facing the SAR sensors to be lower than they should actually be, while they are overestimated around the right side of the building.This layover-caused over-and underestimation, respectively, can be found in all urban areas on the right and left sides of the buildings as depicted in the residual maps (Fig. 4-7).
In contrast, the TanDEM-X and the Cartosat-1 DEMs have almost identical height accuracy in agricultural and forested areas.In forested areas, the mean height of the TanDEM-X DEM is 584.396m, while the mean height of the Cartosat-1 DEM is 581.528m, respectively.In comparison, the mean first-pulse Li-DAR height is 583.312 m.This indicates that the mean canopy height is slightly overestimated by TanDEM-X, which is probably caused by the location of the X-band SAR phase scattering center in the upper part of the vegetation volume (Fig. 8).In contrast, the Cartosat-1 mean height is lower than the first-pulse Li-DAR height.In the agricultural areas, the opposite phenomenon appears: Here, 91% of the first pulse and second pulse heights differ by less than 5 cm (i.e. the first and last pulse almost coincide with height of 577.575 m) while the mean height of Cartosat-1 and TanDEM-X DEM are 578.018m and 577.484 m respectively.Last, but not least, as shown in Fig. 9, Cartosat-1 definitely estimates the height of the lake area better than TanDEM-X which is due to the low coherence because of specular reflection and temporal decoration.
Generally, the results of fusion based on height error maps delivered with the TanDEM X data and the standard deviation maps of the Cartosat 1 stereo matching confirm the improvement of TanDEM-X in all subsets.The quality of the fused DEM is even slightly better than the Cartosat-1 DEM accuracy.Nevertheless, the results demonstrate that using appropriate weight maps, which can precisely identify the quality of the InSAR and the optical DEM will lead to a better fusion result.Therefore, a more sophisticated approach which is be able to estimate the height residuals based on pre-extracted spatial DEM features and using NN was proposed for weight map prediction in the target area where two DEMs would be fused together.
The uncertainty assessment of the fused DEM with using HEMs predicted by NNs shows the increasing the quality of the newly achieved DEM in comparison to TanDEM-X and Cartosat-1 data in the target area.However, using the HEMs of TanDEM-X and Cartosat-1 just produce a DEM with slight corresponding to the quality of Cartosat-1 DEM in the target area, the weight maps derived from the output model of NN can significantly increase the quality of both input DEMs.Besides RMSE and MAE, a decreasing NMAD value as robust metric is evident for this improvement.

CONCLUSION AND OUTLOOK
The uncertainty assessment confirmed that the Cartosat-1 DEM provides higher accuracy than the TanDEM-X DEM over urban areas, while in non-urban subsets both DEMs have almost identical quality.The results of the fusion experiments illustrate that the implemented framework to produce the weight maps of input DEMs can effectively improve the fusion process.In this case of multi-sensor data fusion, where each input dataset comes with different inherent specifications, the weight maps can assist the fusion algorithm to feasibly regard the individual inputs.The strategy of recognizing patterns on the DEMs based on suitable spatial features and corresponding height residual errors in selected subsets as typical areas with different land types makes a general model as predictive weight map for the whole of DEM.
The neural network can be employed to explore the patterns and relationships between the features and height residual map to properly construct predictive weight map model for each DEM.
In the future, we will combine weight map prediction along with more sophisticated DEM fusion approaches like Total Variationregularized DEM fusion.

Figure 1 .
Figure 1.The proposed framework for ANN-based weight map model generation for fusion of TanDEM-X and Cartosat-1 DEMs

Figure 10 .
Figure 10.TanDEM-X height errors: Correlation between RMSE of height residuals and mean HEM value in different subsets

Figure 12 .
Figure 12.The performance of the NNs with increasing the number of Neurons in first layer

Table 1 .
Height accuracy (in meter) of TanDEM-X and Cartosat-1 over different areas

Table 2 .
Accuracies of TanDEM-X and Cartosat-1 DEM fusion based on HEM-weighted averaging (accuracies in meter).

Table 3 .
Results of fusion of TanDEM-X and Cartosat-1 DEM (in meter) by using weight maps generated from different methods