POOR TEXTURAL IMAGE MATCHING BASED ON GRAPH THEORY

Image matching lies at the heart of photogrammetry and computer vision. For poor textural images, the matching result is affected by low contrast, repetitive patterns, discontinuity or occlusion, few or homogeneous textures. Recently, graph matching became popular for its integration of geometric and radiometric information. Focused on poor textural image matching problem, it is proposed an edgeweight strategy to improve graph matching algorithm. A series of experiments have been conducted including 4 typical landscapes: Forest, desert, farmland, and urban areas. And it is experimentally found that our new algorithm achieves better performance. Compared to SIFT, doubled corresponding points were acquired, and the overall recall rate reached up to 68%, which verifies the feasibility and effectiveness of the algorithm.


INTRODUCTION
Poor textural image matching is a challenging task in computer vision, digital photogrammetry and remote sensing field (Hartmann, 2015).Conventional radiometry-based algorithms are effective and robust when encountered with rich textural images.However, the matching result is commonly less than satisfactory when dealing with poor textural images, which requires manual intervention and hinders systematic automation.
Image matching algorithms can be divided into two types.One is based on radiometric information.For instance, normalized cross correlation (NCC) (Gonzalez, 1992), scale invariant feature transformation (SIFT) (Lowe, 2004), distinctive order based self-similarity (DOBSS) (Sedaghat, 2015) are typical radiometry-based.These approach will fail in the presence of ambiguities such as repeated patterns, textures, or nondiscriminative local appearance (Krystian, 2005).Modern algorithms now tend to utilize geometric information to guide the search for correspondence, such as semi-global matching (SGM) (Heiko, 2008), patch-based multi-view stereos (PMVS) (Furukawa, 2007), multiphoto geometrically constrained matching (MPGC) (Li, 2006) etc.In these kind of algorithms interest points are matched under the constraints of geometric relations, their essenc is dual decomposition (Komodakis, 2007), to break down the main problem into smaller problems.Through iterative process all small problems converged to the same or similar solutions.However, focus on poor textural images matching ， few research were conducted.Wu (Wu, 2012) obtained the dense matching point cloud by the combination of point and line features, under the premise of the known image orientation elements.Zickler (Zickler, 2007) took the advantage of PCA-SIFT (Ke, 2004) in highly deformable scene to guide object recognition.
Texture homogeneity always appears as low local contrast.The * Corresponding author intensity-based algorithms are vulnerable to noise interference; therefore, the matching success rate is low.When using featurebased algorithm, not enough feature point can be obtained as well.For texture repeatability, both intensity-based and featurebased algorithm are prone to cause erroneous matches.Also, the texture discontinuity and occlusion leads to distortion of image, even for genuine corresponding points, their feature descriptors often differ greatly.Generally, the homogeneity and repeatability can have severe impact on matching result.In this instance, we draw on graph theory (Chung, 1997;Livi, 2013) for making the match results global optimized in geometry and radiometry, by integrating geometry and radiometry information in a high order tensor (Abraham, 2012) and reach the best assignment matrix via power iteration algorithm (De, 2000).
Image matching problem can be well defined as graph matching problem.Given a set of feature points, the points can be represented by graph node and graph edges encode the relationships between two points.And the problem transforms into finding the correspondences of nodes between two graphs.At present, graph matching mainly have 3 kinds of geometric constraints: unitary constraint，pairwise constraint and triplet constraint.First-order methods only use unitary constraint, like SIFT only consider similarities based on local appearance.Second-order methods, for instance SGM (spectral graph matching) (Leordeanu, 2005), GA (graduated assignment) (Gold, 1996), PGM (probabilistic graph matching) (Zass, 2008), BGM (balanced graph matching) (Schölkopf, 2006), consider pairwise relationship such as distance between feature points.However, second-order methods are distance invariant but not for scale invariant.To overcome the limitation of second-order method, higher order graph matching was invented.HOGM (high order graph matching) (Duchenne, 2009) is a typical representative.It uses the similarity of the angles belong to the triangles formed by each triple nodes between two graph.And by using triplet constraint, the description can be affine invariant.But, when raw data contains numerous gross errors, the methods mentioned above perform poorly.For this purpose, it is proposed an edgeweight strategy to improve graph matching algorithm to obtain well distributed and highly reliable matching points.

METHODOLOGY AND WORKFLOW
In this paper, an improved graph matching algorithm is proposed.In order to reducing the ambiguities and improve matching reliability, both radiometric and geometric information are synthetically utilized by an edge-weight strategy.The approach proceeds as follows: first, a feature point set P is computed using the UR-SIFT (Sedaghat, 2011) algorithm (uniform robust scale invariant feature transform), which extracts the uniformly distributed feature points in the source image, and the SIFT feature points are obtained by a lower contrast threshold in target image.Then the ANN (Indyk, 1998) (approximate nearest neighbor) algorithm is adopted to estimate k tie points in target image corresponding to every single feature point in source image, obtaining a potential tie points set (denoted as Q).Next, graph P G and Q G are constructed by P and Q respectively, and the edge weighted high order affinity tensor A of the two graphs can be built simultaneously.At last, the Power iteration algorithm is applied to find the tensor's leading vector, which is also the nodes assignment matrix Z of the two graphs, thereby the correspondences of the feature points can be constructed.

Figure 1 The workflow of graph based image matching
As can be seen from the Figure 1，the key steps of the running process are : UR-SIFT and SIFT features extraction, affinity tensor A construction, soft assignment matrix Z acquisition, and the discretization of Z to obtain hard assignment matrix Z*.In this section, it focuses on the Construction of feature point sets and edge weighted HOGM algorithm.

Construction of feature point sets
For using geometric constraints in matching, well-distributed feature points are needed.As shown in Figure 2 ), constructed by the feature points extracted in source image and target image respectively, are approximate similar triangles (also been called matched triangles).But if any two vertices of are too close, slight distance variations would change the interior angles of the triangle violently.In this situation, the matched triangles may not be similar triangles.Besides, the triangle constraint become valid in a small region, because two central projection image patches take at different locations with the same scene are local similarity transformation, so the distance between any two vertices of the triangle should not be too long.In addition, in many applications such as aerial triangulation and remote sensing imagery registration, only small amount of tie points are needed, which would better be uniformly distributed.To obtain well-distributed, stable and quantity controllable features, UR-SIFT algorithm is adopted in source image, meanwhile, SIFT with lower contrast threshold is adopted in target image to guarantee feature repeatability.Suppose the feature point set P contains P n feature points, then every single feature point in P can find k (k should be greater than 2) potential correspondences in target image, all these potential correspondences form the feature point set Q, which

Edge weighted HOGM
The essence of image matching is mapping two feature sets.
Graphs(donated as P G and Q G ) can be constructed by given point set P&Q. Graph nodes represent the attributes of the feature points such as gray level and locations, while the graph edges encode their spatial relations such as distances, angles and so on.So the mapping can be cast as the correspondences of the nodes of two graphs, this is the very solution of graph matching.However, the graph matching result is vulnerable to outliers.In this paper, it introduces an edge weighted high order graph matching algorithm to improve matching results when confronted with vast scale of false candidates (outliers).For simplicity, the order of the tensor is restricted to 3.
The 3-oder tensor is given by As illustrated in Figure 1, the triangle descriptor is constructed by the interior angles of the triangle, this is, ) , , ( . The 3-order graph matching problem is to find the optimal solution of formula ( 2) A encodes the relationships and attributes of feature points, * Z expresses the correspondences of the graph nodes.It is noteworthy that 1-order, 2-order and 3-oder graph matching can interconvert to each other in some trivial cases (more details please refer to (8)).
If there exists nodes (that is to say, neither node and q V are outliers, otherwise they are inliers.A major challenge in real-world graph matching problems is to tolerance the numerous outliers arising in typical visual tasks such as image matching and object recognition, sometimes the outliers are more than inliers and that is hard to distinguish inliers from outliers due to clutters.This lead graph matching to local optima and erroneous image matching results.To tackle real-world image matching tasks, it is proposed an edge weighted high order tensor graph matching algorithm. In formula (1), if numerous outliers present in both feature point sets, then the 3-order affinity tensor may contain irrelevant information produced by outliers, so the 3-order affinity tensor A equals to a correct affinity tensor A ~, which produced by point sets have no outliers, plus a turbulent tensor A Δ created by point sets have no inliers and noise (3) Noise is neglected because it has less impact on graph matching than outliers.The main idea of reining the outliers is increasing 4) is a typical iteration step in power iteration of 3order tensor.In general, the elements of initial assignment matrix Z in (4) are set to 0.5 in the beginning of power iteration, and (4) also can be illustrated as Figure 3 Figure 3 sketch of power iteration in the perspective of nodes, the pink circles are outliers As can be seen in Figure 3, the relationship(whether they are matched pair or not) of 1 etc.), i.e. the assignment relationship is determined by the opposite sides of the common vertex.So in graph matching, the critical factor is edges in graph.Formula (4) also can be modified into another form As can be seen in formula (5)，the relationship of edges are determined by graph nodes, and it is more straightforward in Figure 4 Figure 4 sketch of power iteration in the perspective of edges, the pink circles are outliers As shown in Figure 4, similarly to node perspective, the assignment relationship of two edges are determined by triangles which have one common edge in each graph (for example, the assignment relationship of 1' are partial determined by the sum of node similarity 3 there is no prior information of the nodes affinity, then the nodes from two graphs are half-matched, that is to say every element in Z is set to 0.5.So the edge similarity can be defined as Formula ( 6) can be justified by qualitative analysis.If the edge

EXPERIMENTS RESULTS
This section presents evaluation on the proposed EW-HOGM algorithm, three typical poor textural images are used to verify the robustness of the method.Figure 5  In these experiments, UR-SIFT and SIFT are adopted in source image and target image respectively.To tolerate scale and rotation differences, only unary similarity (Euclidean distance of SIFT descriptors) and triplet similarity (Euclidean distance of triangle descriptors) are considered.In order to improve the precision and sparsity of the 3-order affinity tensor, certain elements of the tensor are set to zeros if the unary or triplet similarity is greater than an empirical threshold (for the normalized SIFT and triangle descriptors, the thresholds are set to 0.5 and 10 -2 respectively).In control experiments we use SIFT algorithm, firstly extracts UR-SIFT features in source image and SIFT features in target image (more details please refer to section 2.1), then uses ANN and 2 nearest neighbors constraint to search corresponding points, and rejects matches that the distance ratio (the distance ratio of the closest neighbor to that of the second-closest neighbor) is greater than 0.8.The experimental results demonstrate that EW-HOGM performs better than SIFT in poor textural image matching, especially in matching recall rate.EW-HOGM can obtain much more matching points than SIFT.In woodsy aerial images and desert aerial images, the matched point numbers of EW-HOGM are two times greater than SIFT, and in urban aerial images it is ten times more.Although the recall rates of SIFT in rich textural images are better, however, as can be seen from table 1, the recall rate of SIFT in poor textural image matching is less than 30.0%, the EW-HOGM reach to about 70% mainly owes to both geometric and radiometric information are integrated utilized via affinity 3order tensor.Meanwhile, the distribution of the matched points is better than SIFT.

CONCLUSIONS
It is proposed a graph based algorithm that integrates both geometry and radiometry together to address the poor textural image matching problems.In the algorithm, an edge weighted high order graph matching method are applied, and it improves the matched points number and recall rate.The experiments of typical poor textural image matching results shows that the proposed algorithm performs better than SIFT.Because of the tolerance of outliers, this algorithm can also be employed in shape matching and 3D clouds registration where outliers are massive, besides, this framework can also be applied in gross error detection if it is proper modified.

Figure 2
Figure 2 Geometry constraint based on similar triangles determined by the triangles which have one common vertex in each graph(For example, the value of soft assignment same.So the edge similarity can be treated as a weighted factor when constructing 3-oder tensors edges from different graphs are matched, then triangles include the two edges have high probability of similarity, thus the tensor elements related to the edges should be increased, otherwise they should be decreased because triangles contain outliers may have smaller similarity.By using of edge weighted strategy, A ~ is increased indirectly, and thus robust to outliers.The elements of tensor A can be constructed by (8), then (2) can be solved by HOGM (more details please refer to(Duchenne, 2009)), and the hard assignment matrix Z * of P G and Q G can be obtained by greedy algorithm, so as to the matched result of feature point sets P and Q.
Figure 5 Typical poor textural image matching with EW-HOGM (the circles represent the outliers) , the proposed algorithm employs interior angles of triangle as the geometric constraint.It can be expressed concisely as follows.
Table 1 lists the quantitative comparison of SIFT and EW-HOGM in typical poor textural image matching, these stereo images are captured from forest, desert and urban areas in which showed low contrast, repetitive patterns, discontinuity or occlusion, few or homogeneous textures are abundant.