LINE SEGMENTS MATCHING ALGORITHMl C 0 MBINING MSLD DESCRIPTION AND CORRESPONDING P 0 INTS CONSTRAINT

Aiming at stability of line descriptor in the process of line segment matching, MSLD mean-standard deviation descriptor with corresponding points constraint method is proposed.The method is based on close-range images corresponding points matching and line extraction, in the beginning, determined the points which have the closest distance between the both sides of the target lines on the reference images, virtual line is composed by connecting corresponding points on the search image, and the line which intersects virtual line on the search image is defined as the candidate line segment. Then calculating the MSLD description of the straight lines and the candidate lines respectively, the specific construction steps are as followed: (1) gradient direction and normal direction of the straight line should be determined firstly; (2) for each pixel on the straight line, a rectangular area which is defined as Pixel Support Region (PSR) is established along the gradient direction and the normal direction, and the PSR is decomposed into several same size sub-regions in the normal direction; (3) recording each sub-region the gradient vectors of four directions to obtain a four-dimensional feature vector, and the gradient description matrix of straight line L is composed of all sub-regions feature vectors; (4) mean and standard deviation of the description matrix should be calculated by the row vector, then mean and standard deviation vectors should be normalized to obtain the normalized mean-standard deviation description. Finally, the similarity between the target lines and each candidate lines descriptor is calculated based on the Euclidean distance, using the nearest neighbor distance ratio to determine the corresponding line. Typical region image is selected to perform line matching experiment in this paper, results show that the proposed method has great stability and matching accuracy.

such as computer vision, object recognition, image registration and 3D reconstruction (Ok et al, 2012).Compared with points, straight lines include more rich geometry informations, such as length, direction, gray, gradient, etc, specially in man-made environments,and they may be used where occlusions occur.However,line matching is more difficult than point matching because the end points of the extracted lines is not reliable.Besides that, there is not geometrical constraint, like the epipolar, for lines in two images.So existing line matching research focuses on two aspects: on the one hand is to find effective geometric constraints used to determine the matching candidates, the common constraint conditions include triangulation constraint (Wu, et al, 2015;Zhu,et al, 2005;Zhang,et al, 2013;Wu,et al,2012), corresponding points constraint (Fan, et al, 2010;Fan, et al, 2012), homography matrix constraint (Lu, 2010;Lou, et al, 2011), epipolar constraint (Yi, et al, 2008;Chen, et al, 2016;Schmid,et al, 2000) and so on.Those constraints make candidate lines of the search image limited into setting the range of conditions, reduces 11 Corresponding author: Xue Zhang E-Mail: 670010240@qq.comglobal scope matching search time,improves the matching efficiency.Lou et al.(2011) used homography matrix constraint and neighborhood window gray similarity to determine corresponding lines; Liang et al(2014) used corresponding points constraint and neighborhood window gray similarity to determine corresponding lines.However, those methods more focus on geometric constraints and neighborhood gray window information,this directly uses image gray level information similarity method can not obtain reliable matching results for perspective changes larger or texture complexity of the images.On the other hand is the construction of a linear descriptor,linear descriptors become research hot in recent years.Similar to SIFT (Lowe, et al, 2004) used gradient information of points in the linear neighborhood build line descriptor.Wang et al (2009)  So, in this paper, we propose line segments matching algorithm of MSLD description based on the corresponding point constraint.Using linear neighborhood corresponding point to determine matching candidates on the basis on corresponding reliable points, and then building MSLD descriptor on the center of goal lines and candidate lines respectively, achieves great straight line matching results.

2.EXPERIMENTAL PRINCIPLE
MSLD in the paper line matching process as shown in figure 1: 2) corresponding points which the closest distance on both sides of the goal lines to determine the matching candidates, shrink candidate lines search scope; 3) Establishing MSLD descriptor on the goal line and each candidate straight respectively.First, determined gradient direction and normal direction of the straight lines; second, for each pixel on the line segment, its pixel support region (PSR) is defined and then the PSR is divided into non-overlapped sub-regions; Third, Line gradient description matrix (GDM) is formed by characterizing each sub-region into a vector; the last, MSLD is built by computing the mean and standard deviation of GDM column vectors.To calculate Euclidean distances between target lines and all candidate line descriptors respectively, and then using nearest neighbor distance ratio(NNDR) determine corresponding lines.

corresponding points constraint
As shown in figure 2,the points i ......
in  1 .As shown in figure 3, each of the PSR is divided into three sub-regions, M  3 .( , , , ) By stacking the description vectors of all the sub-regions associated with a line segment, a 4M × N matrix called line gradient description matrix(GDM) is formed as: First, to make the descriptor independent of the line length, we compute the mean vector and the standard deviation vector of GDM column vectors: Second, in order to make the descriptor invariant to linear changes of illumination, the mean vector and the standard deviation vector are normalized to unit norm, respectively.Then, by concatenating the normalized mean vector with the normalized standard deviation vector into a single vector, we obtain a line description vector with a dimension of 8M:

MATCHING STRATEGY
Euclidean distance as the matching measures in this paper, if the ratio of minimum and second minimum Euclidean distance NNDR less than the given threshold, candidate line which has the minimum Euclidean distance with goal line as corresponding

EXPERIMENT
In the experiments, we conducted two sets of experiments.1)MSLD description matching method in this paper.2) Gray similarity matching method on the condition with the same constraints.As shown in figure 4, we conducted three groups of close-range images to evaluate the stability and effectiveness of the MSLD description method.Two matching algorithm for line matching results are shown in table 1. Artificial visual interpretation method judged line matching results are correct or not,   This method has low accuracy and spends a lot of time because of nothing constraints.To sum up, the method in this paper is reliable and effective.

5.CONCLUSIONS
MSLD description in this paper is based on each pixel on the lines built pixel domain support and divided into several sub-regions, it is stability by defining level and vertical direction for each sub-region; different from traditional gray correlation, MSLD characterizing each sub-region into a vector formed gradient description matrix, computing the mean and standard deviation of matrix column vectors to make description dimension has no impact on line length and light;Using NNDR matching strategy determine corresponding lines.The experiment proved that the method has great stability and robustness.

Fig. 1
Fig.1 line matching flow chart 1) First of all, the two images are matched by using SIFT operator, and RANSAC algorithm eliminates error matching the results to obtain initial reliable corresponding points, extracts reference and search image lines by LSD algorithm at the same time; corresponding points of reference and search image respectively.Assuming that a goal line L in the search image, as shown in figure 2(a),finding corresponding points u 、 v which have the closest vertical distance on both sides of L. Connecting the corresponding points u 1 、 v 1 in the reference image, as shown dotted line in figure 2(b), line segments which are intersected with dotted line in the reference image, so line segments L 1 、L 2 、L 3 are candidate lines lists as it shown in figure 2(b).(a) search image (b) reference image Fig.2 corresponding points constraint 3.2 MSLD descriptor Given a line segment L according to chapter 3.1, firstly two directions are introduced: the average gradient direction d  of pixels on the line and its anticlockwise orthogonal direction L d .For each pixel on the line L, a rectangular region centered at it and aligned with the directions d  , L d is defined as the PSR.The PSRs of the pixels on the line along the direction L d are denoted as, G 1 、 G 2 、…、 N G assuming L consists of N pixels.In order to give a more distinctive description for the PSR, each PSR is divided into M non-overlapped sub-regions with the same size along the direction d  :

Fig
Fig.3 MSLD construction Counting each sub-region gradient vector of four direction:  0 、  90 、  180 、 Fig.4 (a) Building area aerial close-range images and line matching results.(b)Different light close-range images and line matching results.(c)Rotating angle about  30 close-range images and line matching results.
Table of MN (Matching Number) means total number of line matching, CM (Correct Matching) means the correct number, CR(Correct Ratio) means correct matching rate.International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W7, 2017 ISPRS Geospatial Week 2017, 18-22 September 2017, Wuhan, China can get more corresponding lines、less error and high accuracy; under the condition of same matching strategy with MSLD, although no-constraint MSLD have more corresponding lines error, there are more error lines.