Research progress on feature matching of SAR and optical images

Before obtaining information and identifying ground target from images, image matching is necessary. However, problems of strong pixel noise interference and nonlinear gray scale differences in synthetic aperture radar image still exist. Feature matching becomes a kind possible solution. To learn the research progress of SAR and optical image matching, as well as finding solutions for above matching problems, a summary for feature matching with SAR and optical image is indispensable. By listing three typical methods below, we can discuss and compare how researchers improve and innovate methods for feature matching from different angles in matching process. First method is feature matching method proposed by CHEN Min et. It uses phase congruency method to detect point features. Feature descriptors are based on gaussian-gamma-shaped edge strength maps instead of original images. This method combines both edge features and point features to reach a match target. The second one is SAR-SIFT algorithm of F. Dellinger et. This kind of method is based on improvement of sift algorithm. It proposes a SAR-Harris method and also a calculation method for features descriptors named gradient by ratio. Thirdly, it is feature matching method proposed by Yu Qiuze et. By using edge features of image and improvement of hausdorff distance for similarity measure, it applies genetic algorithm to accelerate matching search process to complete matching tasks. Those methods are implemented by using python programs, and are compared by some indexes. Experimental data used multiple sets of terrasar and optical image pairs of different resolutions. To some extent, the results demonstrate that all three kinds of feature methods can improve the matching effect between SAR and optical images. It can be easier to reach match purposes of SAR and optical images by using image edge features, while such methods are too dependent on the edge features. * Corresponding author The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3/W10, 2020 International Conference on Geomatics in the Big Data Era (ICGBD), 15–17 November 2019, Guilin, Guangxi, China This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLII-3-W10-77-2020 | © Authors 2020. CC BY 4.0 License. 77


INTRODUCTION
Synthetic Aperture Radar (SAR) images can reflect different ground information from optical images. And based on the special imaging model, SAR images are basically free of imaging time and climate constraints. Therefore, SAR can perform effective remote sensing of the Earth in most weather conditions, and is often used in areas and regions where natural disasters occur frequently. (Ardeshir Goshtasby,2005).By processing and fusing SAR images and optical images, it is possible to obtain feature information more efficiently. The matching of SAR images and optical images is the key premise of image fusion. Obviously, it is necessary and meaningful to study the image matching method between SAR images and optical images, which has become a hot topic in the field of image matching research.
Image matching is the process of identifying points of the same name between two (or more) images. According to different matching elements, image matching can be divided into gray-scale matching and feature matching (Lisa Gottesfeld Brown,1992). The gray-scale matching determines the region of the search image that is most similar to the reference image of a certain template window size based on a similarity measure. The feature matching is based on a certain method or model transformation or mapping to reflect and distinguish the characteristics of images, to achieve the same name point acquisition of images (YANG Sheng, LI Xue jun, et al.,2013). The characteristics of images mainly include color features, texture features, shape features and spatial relationship features. (WANG Zhi rui,YAN Cai liang,2011).
Due to the difference in imaging methods, a single gray-scale correlation match can no longer meet the matching target of SAR images and optical images. More and more scholars have begun to explore the feature matching. (L. M. G. Fonseca,B. S Manjunath,1996).

TRADITIONAL FEATURE MATCHING METHODS
The feature matching of images can be divided into three categories: point feature matching method, line feature matching method and region feature matching method. Representative algorithms are: Scale-invariant feature transform (SIFT) method (David G Lowe,2004), Hough operator (R. O. Duda,P. E. Hart,1972) and Maximally Stable Extremal Regions (MSER) operator (Jiri Matas, Ondrej Chum, et al.,2002).

Scale-invariant feature transform algorithm
The SIFT algorithm (David G Lowe,2004) is a kind of computer vision algorithm. This algorithm detects and describes the local features in the image. It searches for local extremum points in the scale space and extracts its position, scale, and rotation invariants. This algorithm was published by David G. Lowe in 1999 and was completed in 2004.
The SIFT algorithm is based on the search of different scale space to detect feature points and to determine the direction of feature points. The characteristics of the feature points found by SIFT algorithm are very prominent. These feature points do not change due to illumination intensity, affine transformation of images, and varying degrees of interference noise. These feature points include points such as the corner points of ground objects, the edge points of objects on the image, the bright spot in the dark area of the image, and the dark spot appearing in the bright area.
The SIFT algorithm is widely used in computer vision to match images or for local target recognition, and is suitable for various applications such as target retrieval, image retrieval, transformation, registration or image tracking.

Hough line detection algorithm
Hough line detection algorithm (R. O. Duda,P. E. Hart,1972) is mainly used to extract geometric shapes (such as lines, circles, etc.) with certain identical features from the image. The basic principle is to use the point-to-line duality to convert the curve in the original image space to a point in the parameter space. The points of the collinearity in the image plane space correspond to the lines intersecting in the feature space. Correspondingly, all the lines intersecting at the same point in the parameter space have collinear points corresponding to them in the image plane space. Therefore, the Hough line detection algorithm converts the line detection problem in the image plane space into the detection problem of the points in the feature space, and completes the detection task by performing simple accumulation statistics in the feature space. The Hough line detection algorithm has strong anti-interference ability and is not sensitive to noise in images. However, the time complexity and space complexity of the algorithm are high, and the length information of the line segment is lost during the transformation process.

Maximally stable extremal regions algorithm
The Maximally stable extremal regions algorithm (MSER)(Jiri Matas, Ondrej Chum, et al.,2002) is a method for performing spot detection in images. Based on the concept of watershed, the MSER algorithm performs binarization on a grayscale image (the gray value is 0~255), and the threshold is incremented from 0 to 255. In all of the binary images obtained, some of the connected regions in the image change little or even without change, and the region is called the maximum stable extremum region.
MSER has the invariance of affine transformation on image gradation, and multi-scale detection can be realized without any smoothing processing, that is, small and large structures can be detected.

RESEARCH ON FEATURE MATCHING OF SAR AND OPTICAL IMAGES
However, the above conventional methods are not ideal for the feature extraction and detection of SAR images.

Research difficulties
The synthetic aperture radar system uses the Doppler shift phenomenon to improve radar imaging. Based on the characteristics of the active microwave imaging method, the reflection rate of the basic resolution unit of the image will contain random scattering of the ground object. In this imaging process, the radar echoes of these ground targets interfere with each other, resulting in signal superposition, causing speckle noise randomly appearing in the SAR images. This kind of strong multiplicative noise is different from the additive radiated noise of optical images, which makes the traditional feature matching method unable to obtain robust and robust features. And this phenomenon brings great difficulties to the matching of SAR images and optical images.
At the same time, the spectral reflectance characteristics of radar images are quite different from those features in optical images. Reflected on the image, there is a nonlinear gray-scale difference between SAR images and optical images. The traditional gray-based image matching method can't solve the problem enough. (CHEN Min, ZHU Qing, et al.,2016).

Research status at home and abroad
Based on the above difficulties, researchers and scholars have done more research. Among them, the improved feature matching method on multi-feature combination and matching similarity metrics is widely used.
Some researchers have suggested reducing the effects of speckle noise on the image by prefiltering or eliminating interference. Other researchers have improved performance by removing some of the invariance steps to improve algorithm. Lv et al (Wentao Lv, Wenxian Yu, et al.,2011)  There are some ways to obtain feature of images with uniform spatial distribution by setting the intensity threshold of images. At the same time, feature area ranges are enlarged to improve the robustness of the feature descriptor, but this also correspondingly causes the indistinguishability of the feature descriptor. Wang et al (Shanhu Wang, Hongjian You, et al.,2012) use the anisotropic scale space instead of the Gaussian difference scale space in the SIFT algorithm to suppress image noises, or adopt multi-view processing, filter processing and other pre-processing, and then perform matching to achieve the purpose of suppressing image noises. However, these methods reduce images resolution and lost images information at the same time.
Considering the potential application of local features of SAR images, based on the process of feature matching, technical improvement and innovation can be made from feature detection, feature description, feature matching and search strategy to obtain better matching results.

THREE TYPICAL ALGORITHMS EXPERIMENT AND ANALYSIS
Three typical algorithms are selected for experiments and indicators. The three algorithms are: feature matching method which are based on gaussian-gamma-shaped edge strength map for SAR and optical images proposed by Chen Min et al (CHEN Min, ZHU Qing, et al.,2016), the sar-sift algorithm proposed by F. Dellinger et al (Flora Dellinger, Julie Delon, et al.,2012) and matching method proposed by Yu Qiuze et al for SAR images and optical images using modified hausdorff distance and genetic algorithms (YU Qiuze, CHENG Hui, et al.,2006). The first matching method is based on multi-feature combination and similarity measure optimization algorithm. The second algorithm is based on improved SIFT algorithm for feature detection and search space optimization. The last matching method improves matching effect by optimizing match measures and search strategies. The method proposes to eliminate the overall rotation deviation and resolution difference between images after image coarse correction. After logarithmically transforming the SAR image, the method uses phase consistency feature point extraction method (Peter Kovesi,2003) to obtain robust noise suppression. Feature points. Based on this, the method uses on the Gaussian-Gamma-shaped (GGS) bilateral window operator (Peng Lang Shui,Dong Cheng,2012) and obtains the edge intensity map of SAR images and optical images based on two images. SIFT feature description based on edge intensity maps of two images. Finally, the matching process between the SAR image and the optical image is completed based on the constraints of the image geometric condition.

SAR-SIFT algorithm
The SAR-SIFT algorithm is an operator that extracts local features from SAR images and performs feature description. This algorithm has potential applications for all image comparison applications that require stability and invariance, including matching, change detection and template-based target detection. In this method, F. Dellinger et al. proposed a new SAR-Harris feature point detection method. The SAR-Harris feature detection method uses multi-scale SAR-Harris matrix and multi-scale SAR-Harris function to calculate the multi-scale results of the original image, thus replacing the scale space in the SIFT algorithm. The feature points acquired by the SAR-Harris feature detection method are mostly located at the corners or bright spots of the image. The SAR-SIFT algorithm also proposes to apply fuzzy Gaussian functions with different weights in different directions of SAR images. Using the ratio of average ratio (ROA) (Ridha Touzi, Armand Lopes, et al.,1988), A new gradient calculation method named Gradient by Ratio (GR) is defined. The GR histogram is used to calculate the gradient histogram and gradient direction of the SAR image, so that the detected feature points are characterized and the corresponding ratio feature descriptors are obtained. (Ratio Descriptor).

Matching SAR and optical images using modified hausdorff distance and genetic algorithms
This method performs matching of SAR and optical images by using edge features. Considering the coherent noise interference of SAR image, the SAR image is subjected to logarithmic transformation preprocessing, and the canny edge feature map (J.F. Canny,1986) of the preprocessed SAR image is extracted, the Canny edge feature map is directly extracted from the optical image. Based on the Canny edge feature, the method uses genetic algorithm to search the template of the image, and calculates the modified hausdorff distance of SAR and optical images to complete the feature matching. The experimental image is shown in Figure 1

Comparison and analysis of experiment
Based on experiment, there are comparison of some index in three kinds of feature matching methods as follows table 2. Feature matching method based on gaussian-gamma-shaped edge strength map uses the phase consistency feature extraction algorithm to acquire feature points, and performs feature description based on the edge features of the image. When the edge features of the image are insufficient, this method is not effective based on the feature description of the edge features. At the same time, the use of linear constraints for feature matching calculation improves the matching accuracy of the algorithm, but it increases the time complexity of the algorithm.
The SAR-SIFT algorithm uses the SAR-Harris feature point detection method to obtain fewer optical image features. Moreover, most of the feature points acquired by SAR-SIFT algorithm are corner points, which are not good for flat areas.
The method using modified hausdorff distance and genetic algorithms to perform matching search increases the running speed, but it will be limited by the edge features, and the matching measure has limited precision.