COMPARING STEREO IMAGE MATCHING PERFORMANCE BY MULTI-DIMENSIONAL SEARCH WINDOWS

Image matching is a key technology for extraction of dense point cloud and 3D terrain information using satellite/aerial imagery. In image matching using brightness values of pixels, the size of search window is an important factor for determining the matching performance. In this study, we perform matching using multi-dimensional search windows applicable to area-based matching and compare the performance. Also, the search window is reconfigured by using the linear information existing on the image, and the matching is tried. Comparing the fixed search window and the multi-window matching results, it was confirmed that the multiple windows under the same conditions show relatively high accuracy. We can also see that the method of applying the line element has slightly better accuracy. As a result of applying the line element extraction technique, a large number of pixels are not extracted compared with the total image pixel amount. There was no significant difference in the results of visual analysis. However, we have confirmed that this technique has contributed to improving accuracy. * Corresponding author


INTRODUCTION
The 3D information obtained from the image is extracted to DSM (Digital Surface Model) or point cloud, and it can be used as an image map or a 3D object model.Image matching is a key technology for extraction of dense point cloud and 3D terrain information using satellite/aerial imagery.
The most ideal image matching is to find the corresponding points of individual pixels of all images.However, an actual pixel has very limited brightness value information.Therefore, it is very difficult to find a matching point using only information of one pixel.As a representative method for dense matching, there is an area-based matching technique.This technique computes each pixel's disparity independently over a local support region.Area-based matching uses the brightness values around one point and uses the correlation coefficient or entropy of these points.This technique has advantages of fast computation, but it has a disadvantage that accuracy is unstable.Therefore, for exact matching, constraints must be set strictly through pyramid image-based matching or by limiting the range of epipolar lines.Global method solves the stereo problem in an energy minimization process.This technique is accurate but slow due to time-consuming global optimize.
To compensate for the disadvantages of each technique and to obtain the decent image matching, the SGM (Semi Global Matching) technique including the advantages of area-based matching and global-based matching is used (Hirschmüller, 2008).SGM uses mutual information based on the brightness value information.This technique also cannot ignore area-based pixel information in that it computes the matching cost around the pixels.
In image matching using brightness values of pixels, the size of search window is an important factor for determining the matching performance.Rhee et al., 2016 reported that dense image matching can be achieved by using only local area-based matching through previous studies, and proposed a technique using multi-dimensional search window and global optimization method In this study, we perform matching using multi-dimensional search windows applicable to area-based matching and compare the performance.Also, the search window is reconfigured by using the linear information existing in the image, and the matching is tried.

PROPOSED METHOD
While there are many matching schemes proposed, this paper examines the capability of an area-based matching method for the task.Area-based matching is a technique to set a search window in reference image and to find the most similar area in target image.The simplest way to set the search window is to extract the image patch of a fixed size and determine the maximum correlation with the target image In this paper, point cloud and DSM were extracted by matching the image with from multi-dimensional search windows and by the conventional fixed-size window.In order to improve the performance of image patch extraction, we added a method to extract the patch by rotating the patch along linear features existing in the original

MULTI-DIMENSIONAL SEARCH WINDOW
The size of search window for calculation of similarity is very important.Objects such as buildings existing in the original image have different sizes and different amount of texture information.Resolution of the image is different according to the platform for acquiring the image, it is difficult to obtain a stable matching result when using a fixed search window.If the search window size is large, matching result is a reliable, but it is difficult to express detailed representation.If too small, it is possible to recover detailed representation, but outlier will also be increased.
To solve this problem, research on matching with adaptive and variable search window setting has been performed.Rhee et al., 2016 used a multi-dimensional search window for stable and accurate matching.Multi-dimensional search window consists of many set of a full, upper left, upper right, lower left, lower right windows according to various window sizes.Compared to a fixed window case, many similarity comparisons can be performed.Correlation is calculated at a target point within the search range for each search window, and if there is a search point having the highest correlation coefficient among the search windows having the same ID, the cost is increased.We call this a hit.We determine the search point that has obtained the greatest number of hits as the matching point.

EXTRACTION OF LINE COMPONENTS WINDOW
The image information includes the roof information of buildings, linear information such as roads, and texture information.This is the main information that cannot be acquired by the Lidar sensor.When this information is utilized, it is possible to use not only an improvement of matching performance but also a structural data necessary for generation of a 3D model.
In this study, it was expected that higher performance image matching would be possible by using linear information among geometric information existing in image information.As a method of extracting a search window using linear information, there was a method of applying an image mask around a line.(Li, 2004) As the line extraction method, Burns et al., 1986 and LSD (Line Segment Detector) algorithm (Gioi et al., 2012)   In this study, we tried to improve the patch performance by using the rotation angle which can be deduced from the linear information.First, line extraction is performed on the epipolar image and it is determined whether a line exists in the search window at the time of extracting the image patch.When there is a line in the centre of the search window, the angle of rotation of the line on the epipolar line is calculated, and the patch is rotated so that the line component coincides with the epipolar line.The patch extraction method considering rotation is as follows.The test image had a GSD of about 0.7m.In order to verify the experiment, DSM was constructed by interpolating the Lidar data acquired at intervals of 0.5m and used for analysing the vertical error of the extracted DSM. Figure 5 shows the reference DEM images derived from the reference height data.

EXPERIMENTS AND ANALYSIS
The image matching method basically applied the method of grey-level scale and converted the target image into the epipolar image to perform the matching on the epipolar image.Experiments were performed with three fixed search windows set and multi-dimensional search windows, the size of the multiple search window is set from 5 to 13 pixels, 25 of 5 levels in total.The fixed search window was set to 5x5, 9x9, and 13x13 patches, and the region with the highest correlation coefficient on the epipolar line was selected.The matching was processed at every pixel of the 1.0 scale image domain.A series of tasks were performed with in-house software developed for image matching.
Results analysis was performed by visual analysis and quantitative analysis by comparing height values with lidar data.Comparisons of extraction points were performed at every point in the generated DSM.Post-processing and interpolation were not carried out for comparison of the matching performance itself.The results of the performance comparison between fixed patch size and Multi-dimensional image patch are shown a Figure 6.Accuracy analysis was performed by applying AVG, STD, RMSE and NMAD to the difference between Lidar data and corresponding point.As can be seen from Table 1, all numerical values show lower error rates when using the multidimensional search window technique under the same conditions.Table 2 and Figure 7 shows the extracted accuracy when using lines.We can see that the method of applying the line element has slightly better accuracy.As a result of applying the line element extraction technique, a large number of pixels are not extracted compared with the total image pixel amount.
There was no significant difference in the results of visual analysis.However, we have confirmed that this technique has contributed to improving accuracy.

CONCLUSIONS
In this paper, we describe an improvement of the search window setting technique for applying the image matching technique using pixel brightness value.We have also confirmed that the proposed method can successfully extract the point cloud by applying the proposed multi-dimensional search window scheme to the area-based matching rather than using a simple matching window.The multi-dimensional window technique is not limited to correlation-based matching.In addition to correlation, it is expected that it can be applied to most matching methods that extract image brightness values such as entropy and texture information as image patches.This technique is concerned with slowing down to a large amount of computation.Therefore, an appropriate area search method should be performed together to supplement this.
The results of the patch extraction using the line also showed some improvement in accuracy, but the performance did not appear to be significant.However, this method showed the possibility of extracting the line component existing in the image as a 3D element.The matching method using line extraction is expected to be highly applicable in that it can obtain 3D information of pure line elements.

Figure 1 .
Figure 1.Multi-Dimensional Search window (5 step) are used.The line extraction result using each technique is shown in the following figure.

Figure 2 .
Figure 2. Comparison of line extraction method The line extraction results show that Burns is suitable for extracting multiple lines, and LSD has a strength in building boundary representation.In this experience, the extraction of linear features was used by LSD technique.

Figure 3 .
Figure 3. Patch extraction method considering rotation

Figure 5 .
Figure 5. Reference DEM derived from LIDAR data for Terrassa

Table 1 .
The performance comparison between fixed patch size and multiple image patch

Table 2 .
The performance comparison with multi-dimensional search window using line elementsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4, 2018 ISPRS TC IV Mid-term Symposium "3D Spatial Information Science -The Engine of Change", 1-5 October 2018, Delft, The Netherlands