AUTOMATIC SHAPE-BASED TARGET EXTRACTION FOR CLOSE-RANGE PHOTOGRAMMETRY

In order to perform precise identification and location of artificial coded targets in natural scenes, a novel design of circle-based coded target and the corresponding coarse-fine extraction algorithm are presented. The designed target separates the target box and coding box totally and owns an advantage of rotation invariance. Based on the original target, templates are prepared by three geometric transformations and are used as the input of shape-based template matching. Finally, region growing and parity check methods are used to extract the coded targets as final results. No human involvement is required except for the preparation of templates and adjustment of thresholds in the beginning, which is conducive to the automation of close-range photogrammetry. The experimental results show that the proposed recognition method for the designed coded target is robust and accurate.


INTRODUCTION
Close-range photogrammetry (CRP) is a key technique for determining the three-dimensional (3D) geometry of a physical object by analyzing and measuring features appearing in twodimensional photographs of the object and has found many diverse applications in the fields of industry, architecture, automotive, as well as virtual reality technique (Luhmann, 2010).In CRP, artificial coded target is the hardware basis and applied widely so as to obtain object world coordinates to establish the object space coordinate system.According to the structural features, coded targets can be classified into three categories, 1D target, planar target and 3D target.Among them, the planar coded target is in good graces of many researchers for the merits of simple manufacture, low cost and convenient maintenance.Schneider, 1993;(g) Van Den Heuvel, 1993;(h) Ma, 2006; Until recently, the planar coded target commonly used in CRP contains concentric circles, circular dots and other types, shown in Figure1.In order to correctly reconstruct the corresponding relationship between 2D image coordinates and 3D world coordinates, it is critical to accurately extracting the image coordinates of feature points (Feng et al., 2002).
To implement such procedure, large quantity work has been done.Ahn et al. (2001) proposed a circular coded target for automatic image point measurement and identification.Through all the processing stages, ellipse fitting played a significant role in recognition the dots around the central concentric ring.Cronk et al. (2006) presented the practical developments of manual and semi-automatic hybrid measurement using the red circle-based coded targets.Bethmann et al. (2010) presented a least-squares matching algorithm using the plane projective transformation model and polynomial transformations to handle geometric distortions between the images with coded targets.Fraser et al. (2009) discussed a hybrid measurement to perform semiautomatic extraction of feature points of circular dots target in order to determine image point correspondence for image matching.In addition, Luhmann (2014) discussed a feature of projective geometry which causes eccentricity in the image measurement of circular targets.
In general, a circle becomes an ellipse after projective transformation.For an ellipse, its simple geometric features and explicit mathematical expression make it easy for detection and furthermore it is not common in natural scenes.Hence, circlebased coded targets are increasingly popular in CRP.And the common algorithms to extract circle-based coded targets are mainly based on ellipse fitting.
However, due to discretization error caused by noise, illumination and shooting viewpoints, the parameters of ellipse might be incorrect when the number of pixels on eligible edges is not enough to accurately determine the shape of the ellipse.Thus the detected feature point coordinates estimated by central ellipse is unreliable and is not suitable to use for decoding the target.
In order to overcome the shortages of circle-based coded target extraction methods mentioned above, this paper presents a novel design of circle-based coded target, including feature point coordinate and feature point serial number simultaneously.This target design not only can fulfill the need of being detected and recognized accurately, but also could be located from farther distance and wider angle, especially in complex environment.
Based on the features of proposed target, a comprehensive method to perform robust extraction of coded targets is introduced.The method utilizes a coarse-fine procedure: firstly, instead of using ellipse fitting, shape-based template matching is creatively applied to obtain the approximate regions of coded targets; then, center points of coded targets are determined by region growing; finally, false results are filtered by parity check.

METHOD
The organization of proposed method is shown in Figure 1.This method has three main stages: 1) template preparation by a set of geometric transformations based on the original target; 2) shapebased template matching; and 3) center points location based on region-growing and false results removal by parity-check.

Design of the coded target
The novel design of planar coded target is shown as follows: Figure 3.The design of the proposed coded target In Figure 3, there are two main parts in the designed coded target.The upper part is used for extracting the feature point coordinate, named "target box".From outside to inside, there are four different patches, black rectangle, white ring, black outer circle and white inner circle while the center points of latter three patches are the same.Specifically, the central white circle is utilized to be located precisely by total station instrument.And the lower part of white ring is covered by a black rectangle which makes an indentation of the ring.With this creative design, the coded target can own an advantage of rotation invariance as well as offer orientation information in the step of template matching.Moreover, the shape features made by white-black patches could be identified even from a far distance and wide angle as long as the imaging distortion is within acceptable limits.In addition, based on empirical knowledge, the concentric-circle-based shape rarely appears in natural scenes which could substantially decrease the disturbance from background.
The feature point serial number is expressed by the lower part, named "coding box".Owing to the random location of artificial target and measurement system, it is necessary to re-identify the feature point set in order to completely match 2D target image coordinates and 3D target world coordinates.However, for most of circle-based coded target used in CRP, the coding box is commonly designed to be uniformly distributed along the circumference, which is likely to reduce the robustness of target extraction algorithm.Therefore, the target box and coding box is totally separated in our design.Meanwhile, coding box also plays a role of "check code" and could be used to remove false results to some extent.The details of this part will be introduced later.
In this paper, the target is almost the half size of A4 paper and it can be scaled in equal proportion if necessary.

Template preparation
Here, templates are created based on the proposed target.However, original target will be distorted by perspective projection transformation during imaging.Thus it is essential to perform some pre-processing, as follows: 1. Use the original target box as the basis.
2. Select a region small than black rectangle but bigger than white ring and remove the central white circle.
3. Minify the result in step 2 to a designated size (according to the CRP practical application requirements, such as 100 × 100 pixels) as the original template.4. Apply a set of geometric transformations to the original template, defined as: (1) 1 0.05 0 , 0,1, ,10 01 where I is the original template for input and C is the template set after preparation.
x Η ,  R , n S are geometric transformations of horizontal scaling, rotating and scaling, respectively.Hence, we can obtain a template set, which contains almost all the possible shape features of coded targets in imaging.

Shape-based template matching
In this part, shape feature is used for template matching.In order to perform shape matching, we use a similarity measure (Steger, 2002) to make a judgment whether the two objects are similar using a threshold.To define the similarity measure, we first define the template of an object as a set of point ( , ) xy .In the matching process, a transformed template must be compared to the image at a particular location by a similarity measure.Then, we sum the normalized dot product of the direction vectors of the transformed template and the search image over all points of the template to compute a matching score at a particular point ( , ) T q x y  of the image.This similarity measure is defined as follows: The normalized similarity measure (5) returns a number range from 0 to 1 as the score of a potential match.A score of 1 indicates a perfect match between the template and the image.
Meanwhile, an image pyramid with 3 levels is used to speed up the matching procedure.Therefore, the detailed steps of template matching based on shape feature could be summarized as follows: 1.For each input image, a slide window with the size as same as the original template is used.The sliding step is set to the half window size.For each move, the corresponding patch of image is defined as a region of interest (ROI).2. For each ROI, bilateral filtering (Tomasi et al., 1998) and Canny edge detection (Canny, 1986) are performed successively so as to enhance edge contour as well as reduce redundant edge points.Consequently, a set of edge points is obtained 3. Using the proposed similarity measure to compute the matching score between ROI and template set t C . 4. Compare the score and predefined threshold m T , if score is greater than m T , the characteristics of the corresponding template and ROI will be stored.

Center points extraction
After the approximate regions of target boxes are detected by template matching, we take the region growing algorithm (Fan, 2001) to find the coordinates of candidate feature points, as follows: 1.For each candidate patch, Otsu's adaptive method is used to perform the process of segmentation.2. The center point of patch is served as the seed if its gray value is zero.3. Using region growing to connect the remaining pixels one by one at the smallest scale.
After the above steps, the minimum patch including the corresponding feature point is determined.Then the coordinate ( , ) ii XY of candidate feature point is defined as:

Filtering by parity check
After the above steps, coding box can help a lot to remove false results.In this part, even check, one of parity check, is used to further eliminate false positives.As mentioned above, there are eight blocks in the coding box.Every block is assigned with two colors, black or white in row-major order and defined as number 0 or 1.Thus it can be regarded as an eight-bit binary number, i.e.To improve the robustness, 7 B is selected as check bit to identify odd-even property while the rest are code bits, as follows: where  is the bit-wise XOR operation.

RESULTS AND DISCUSSION
To verify the effectiveness of the proposed method, experiments are conducted with images in scenes.These images are captured from different distances, viewpoints and contain some proposed targets as well as other objects, which make the background much more complicated.The coded targets are printed by a common inkjet printer (HP P1606dn) and a Sony NEX-7 digital camera with 50mm lens is used.In addition, practice results indicate that, when the target size in imaging is range from 20 to 100 pixels, it is more advantageous to the digital image processing and the realization of the proposed algorithm.
Programming was done in C++ on a Window 7(64 bit) platform.
The hardware system is an off-the-shelf computer with 4GB RAM and a dual-core 3.4 GHz CPU.Two of the experimental results are shown in Figure 5 and Figure 7.It can be seen that all the coded targets are correctly identified after applied the proposed algorithm (labelled with red crosses).Table 6 and Table 8 present the corresponding quantitative assessment of the location accuracy of feature points.Here, the ground-truth is determined by eye-measurement.

Figure 2 .
Figure 2. Flowchart of the proposed target extraction algorithm

y
are the extreme points in four directions of patch boundary.

Figure 4 .
Figure 4. Parity check for coding box

Figure 5 .
Figure 5. Identification and location of the coded targets (viewpoint: 60°) The direction vectors can be generated by a number of different image processing operations, e.g., edge, line, or corner extraction.The search image can be transformed into a representation in which a direction vector ,