REGISTRATION OF OPTICAL AND SAR SATELLITE IMAGES BASED ON GEOMETRIC FEATURE TEMPLATES

: Image registration is required for different remote sensing applications, like change detection or image fusion. Since research studies have shown the outstanding absolute geometric accuracy of high resolution radar satellites images like TerraSAR-X, the importance of SAR images as source for geolocation enhancement has increased. Due to this fact, multi-sensor image to image registration of optical and SAR images can be used for the improvement of the absolute geometric processing and accuracy of optical images with TerraSAR-X as reference. In comparison to the common optical and SAR image registration methods the proposed method is a combination of intensity-based and feature-based approaches. The proposed method avoids the direct and often difﬁcult detection of features from the SAR images. SAR-like templates are generated from features detected from the optical image. These templates are used for an intensity-based matching with the SAR image. The results of the matching process are ground control points, which are used for the estimation of translation parameters followed by a subpixel translation of the optical image. The proposed image registration method is tested for two pairs of TerraSAR-X and QuickBird images and one pair of TerraSAR-X and WorldView- 2 images of a suburban area. The results show that with the proposed method the geometric accuracy of optical images can be enhanced.


INTRODUCTION
Image registration is an on-going research topic and required for different applications in remote sensing, like change detection or image fusion. Since research studies have shown the outstanding absolute geometric accuracy of high resolution radar satellite images like TerraSAR-X (Ager and Bresnahan, 2009), the impact of SAR images as source for geolocation enhancement, especially for image registration, has increased. Multisensor image to image registration is still a challenging task especially in the case of optical and SAR images. Core differences in the sensor geometry and radiometry of the optical and SAR sensors are challenging for the registration of the images acquired by such sensors.
An important application of optical and SAR image registration is presented in the work of Reinartz et al. (2011) and Perko et al. (2011). In these papers, the absolute geometric processing and accuracy of optical images are improved by using ground control points (GCPs) selected from high resolution TerraSAR-X reference images. These papers show how the precision of high resolution SAR images can be used to improve the geometric accuracy of optical satellite data.
A variety of different image registration approaches were developed over the last years. The most common registration approaches can be classified in two categories. The first category are intensity-based registration approaches. For instance, mutual information is a widely used intensity-based registration approach and investigated in different papers like Chen et al. (2003) and Suri and Reinartz (2010). Further intensity-based registration methods are using the cross-cumulative residual entropy (Hasan et al., 2012) or the normalized cross-correlation coefficient (Shi et al., 2012). The second category are feature-based registration approaches, which are commonly based on the detection and The above described methods for optical and SAR image registration have several drawbacks. Intensity-based registration approaches like mutual information often fail because of the different radiometric and geometric properties of the SAR and optical images. The paper of Zhao et al. (2014) shows that reliable results are only achieved for image pairs with small misalignment. A problem of feature-based approaches is the feature detection from the SAR image due to speckle noise and geometric radar effects. Therefore, the quality of the detected features shows strong influence on the results of these registration methods. Nevertheless, over the last years feature-based approaches are proven to be more suitable for the problem of multisensor image registration than intensity-based methods.
The proposed method offers an alternative approach for the task of geometric accuracy improvement of optical images. In comparison to the common optical and SAR images registration methods our method combines feature-based and intensity-based approaches. Furthermore, the direct and often difficult detection of features from the SAR images is avoided at the proposed method. Therefore, the features (roundabouts) are only detected in the optical image. To avoid the problem of an intensity-based matching between optical and SAR images patches, SAR-like templates are generated from the detected features. These templates are used for an intensity-based matching with the SAR image. The result of the matching process are ground control points, which are used for the estimation of translation parameters. Bilinear interpolation is used for a subpixel image translation of the optical image.

METHOD
The proposed method for the registration of optical and SAR images consists of four steps: (1) feature detection and extraction, (2) feature matching, (3) transformation model estimation and (4) image resampling and transformation (Zitová and Flusser, 2003). The whole image registration chain is illustrated in Figure 1 and detailed in the remainder of Section 2. Figure 1: Overview of the proposed method.

Feature Detection and Extraction
In the first step of the image registration chain, spatial features (roundabouts) are detected and extracted from the image. Since the direct detection of features from the SAR images is often difficult, the features are only detected and extracted from the optical image. OpenStreetMap data is used to find the rough location of roundabouts in the optical image.
The edge detection plays a crucial role in the further process. Therefore, it is important to highlight the differences between the street and the vegetation around the street or inside the central island of the roundabout. Therefore, the normalized difference vegetation index (NDVI) image is generated and used for edge detection. The edge image in an area around each roundabout is computed with a zero crossing edge detection method (Lindeberg, 1998). In the end, a circle fitting method is used to detect the boundary and the center of the central island of each roundabout. Figure 2 shows the NDVI image of a roundabout and the corresponding edge image, where the detected boundary of the central island is highlighted in red.

Feature Matching
In the second step, the extracted roundabout geometry from the optical image is used for a template based matching with the SAR image. Therefore, the extracted roundabout information, i.e. center position and radius of the central island, are used to generate SAR templates for all roundabouts. Figure 3 illustrates the comparison between the extracted roundabout from the optical image (a), the generated template (b) and the roundabout appearance in the SAR image (c). Figure 3: A roundabout in the optical image (a), the generated roundabout template (b) and the same roundabout in the SAR image (c).
The generated roundabout templates are then used for an intensitybased matching with the SAR image. From this procedure the center positions of the roundabouts from the SAR image are obtained. Figure 4 shows an example of a roundabout in the SAR image, where the derived center position is marked in red.
For a more detailed insight into the used circle fitting method, the template generation and the intensity-based matching see our previous work (Merkle et al., 2015) and the work of Wu et al. (2008).

Figure 4:
The roundabout in the SAR image and the extracted circle (red) and center position (red+) after the matching process.

Transformation Model Estimation
In the third step, the extracted center positions of the roundabouts from the SAR image are used as GCPs. With these GCPs, transformation parameters are generated to improve the relative position of the optical image with respect to the SAR image. For optical images from high resolution sensors like WorldView-2 or QuickBird, which already exhibit a geometric accuracy of a few meters, a translation or an affine transformation can be used for the registration improvement or for a direct improvement of the given rational polynomial coefficients (Müller et al., 2012). In this paper, a subpixel image translation model is used, which requires only few GCPs with high geometric quality.
For estimating the translation parameters, the distance between the GCPs detected from the SAR image and the roundabout centers detected from the optical image in xand y-direction is calculated (the x-direction corresponds to east and the y-direction to north). Afterwards, the mean values of the distances in both directions are calculated and used as translation parameters.

Image Resampling and Transformation
In the last step of the image registration chain, the estimated translation parameters are used to translate the optical image, and thus, to improve the geometric accuracy of the optical image. Bilinear interpolation is used to resample the optical image after the subpixel image translation.

Data Description
The experimental data set consists of TerraSAR-X, WorldView-2 and QuickBird imagery acquired over Oberpfaffenhofen, Germany (see Table 1). The WorldView-2 and the QuickBird images are orthorectified using the Shuttle Radar Topography Mission (SRTM) DEM. The spatial resolution is 0.5 m for the TerraSAR-X image, 2 m for the WorldView-2 image and 2.4 m for the Quick-Bird image. To cover all roundabouts in the TerraSAR-X scene one WorldView-2 and two QuickBird images are required. A reference dataset is used for the evaluation of the results. The dataset consists of reference points, which have been measured with a Leica GPS 1200 and an absolute geometric accuracy in the range of 30 cm. For every roundabout in the scene 11 to 16 reference points have been measured at the boundary of each central island. The coordinates of all roundabout centers are calculated with an ellipse fitting method (Gander et al., 1994) from the reference points, which are located on the boundary of the central island. This points are needed to evaluate the geometric accuracy of the translated images at the center of each roundabout. Moreover, 6 further reference points (street crossings) are measured to evaluate the geometric accuracy of the translated images apart from the roundabouts.

Results and Discussion
The image registration chain described in Section 2. is tested on two pairs of QuickBird and TerraSAR-X and one pair of WorldView-2 and TerraSAR-X images of a suburban area. The whole scene covered by the images contains five roundabouts with radii between 7.3 m and 24.9 m. Three of the roundabouts are contained in the QuickBird image acquired on 24.09.2011 (denoted by QuickBird image 1) and two of the roundabouts are contained in the QuickBird image acquired on 24.10.2011 (denoted by QuickBird image 2).
The center and radius of each roundabout are detected in all optical images by applying the method explained in 2.1. Afterwards, SAR like templates are generated and used for the intensity-based matching with the TerraSAR-X image, as described in 2.2. The result of the matching process are the circle center positions of all roundabouts in the TerraSAR-X image. These circle center positions are the GCPs, which are required for the estimation of the translation parameters.
In order to determine the translation parameters, the distance between every GCP and the detected roundabout center from the optical image in xand y-direction is calculated. Figure 5 shows the image scenes, which contain the roundabouts of the Quick-Bird images and the WorldView-2 image. The image scenes in Figure 5(a) and (b) show a subset of the QuickBird image 1, the image scene in Figure 5(c) shows a subset of the QuickBird image 2 and the image scenes in Figure 5(d) and (e) show a subset of the WorldView-2 image. The blue vectors are pointing from the roundabout center, detected from the optical image, to the corresponding GCP. In comparison, the red vectors are pointing from the roundabout center detected from the optical image, to the corresponding roundabout center from the reference data. For an simplified visualization, all vectors in Figure 5  Each red and blue vector pair in Figure 5 (a)-(c) is pointing approximately in the same direction and has almost the some length. Moreover, the different vector pairs within each QuickBird image have very similar directions and lengths. An improvement of the geometric accuracy of the QuickBird images is assumed if each QuickBird images is shifted towards the average direction of the corresponding blue vectors. In comparison, in Figure 5 (d) and (e) the red and blue vectors differ in their length and direction within each vector pair and within the whole WorldView-2 image. By shifting the WorldView-2 image in the average direction of the blue vectors, the geometric accuracy is improved for some points but not for the whole image. A possible reason is that the geometric accuracy of the TerraSAR-X image is not significantly higher than the geometric accuracy of the WorldView-2 image. Therefore, further results are only shown for the case of TerraSAR-X and QuickBird image pairs.
For estimating the translation parameters the mean value of the distances in xand y-direction between the GCPs and the roundabout centers from the optical images are calculated. In particular, the QuickBird image 1 is translated by 6.65 m in xand by −2.16 m in y-direction and the QuickBird image 2 by −4.98 m in xand by 2.88 m in y-direction. After translating both QuickBird images, the geometric accuracy improvement can be evaluated. Table 2(a) and (b) show the distance in xand y-direction and the Euclidean distance between the detected roundabout centers of the QuickBird images and the reference dataset. Furthermore, the mean deviation and the root mean square error (RMSE) is calculated. In contrast, Table 2(c) and (d) shows the distances between reference points detected from the QuickBird images and from the reference dataset, together with the mean deviation and the RMSE.
The geometric accuracy enhancement of both QuickBird images can be deduced from Table 2 case of the QuickBird image 2. Although a simple subpixel translation is applied on both QuickBird images, the improvement of the accuracy is also measured at the reference points apart from the roundabouts. In the QuickBird image 1 the geometric accuracy of the two reference points is enhanced from 2.55 to 1.18 pixel and the geometric accuracy of the four reference points in QuickBird image 2 is enhanced from 3.09 to 1.18 pixel.
An illustration of the impact on both QuickBird images related to the translation can be seen in Figure 6. The chessboard overlays in Figure 6 confirm that with the proposed image registration chain also the alignment of both Quick-Bird images to each other is improved. In both example scenes a substantial improvement of the alignment of both images can be clearly distinguished.

CONCLUSION AND FUTURE WORK
In this paper a new approach for image registration of optical and SAR satellite images is presented. By combining classical feature-based and intensity-based approaches the proposed method avoids the drawbacks of them. Due to speckle noise and geometric radar effects, the direct detection of features from SAR images for feature-based approaches is often difficult. Therefore, the proposed method detects features only from the optical image.
Intensity-based approaches are suffering from the differences in the sensor geometry and radiometry between the two sensor. Due to this, our method generates SAR-like templates from detected edges, and uses them for an intensity-based matching with the SAR image.
The experiments confirm that by applying the proposed method on optical and SAR satellite images pairs an improvement of the geometric accuracy of the optical image is achieved. Particularly in cases, where the geometric accuracy of the SAR image is significantly higher than of the optical image, the improvement of the geometric accuracy of the optical image is distinct. In fact,  The distances dx and dy in xand y-direction and the Euclidean distances dx,y between the detected roundabout centers (reference points) from the two QuickBird images and from the reference dataset, the mean deviation, and the root mean square error (RMSE). The x-direction corresponds to easting and the y-direction to northing coordinates.
only with two or three GCPs in one image and a subpixel image translation, the geometric accuracy of the optical image is improved. Additionally, the alignment of different optical image scenes with a small overlapping area is achieved with the proposed method.
In the future the extracted GCPs will be used for a direct improvement the sensor model parameters. Furthermore, the optical image will be orthorectified by using the improved sensor model parameters and the digital elevation model (DEM) to cover geometric distortions, and not only a translation in image space.