Aircraft segmentation in SAR images based on improved active shape model

In SAR image interpretation, aircrafts are the important targets arousing much attention. However, it is far from easy to segment an aircraft from the background completely and precisely in SAR images. Because of the complex structure, different kinds of electromagnetic scattering take place on the aircraft surfaces. As a result, aircraft targets usually appear to be inhomogeneous and disconnected. It is a good idea to extract an aircraft target by the active shape model (ASM), since combination of the geometric information controls variations of the shape during the contour evolution. However, linear dimensionality reduction, used in classic ACM, makes the model rigid. It brings much trouble to segment different types of aircrafts. Aiming at this problem, an improved ACM based on ISOMAP is proposed in this paper. ISOMAP algorithm is used to extract the shape information of the training set and make the model flexible enough to deal with different aircrafts. The experiments based on real SAR data shows that the proposed method achieves obvious improvement in accuracy.


INTRODUCTION
As the development of SAR technology, imaging technology becomes much more mature. High resolution SAR images bring much convenience to automatic target recognition (ATR), because more structure and geometry details can be displayed. This makes aircraft recognition possible. Aircraft targets have great value and always arouses much attention in image interpretation. There has been some effective methods of aircraft detection and recognition (F. Zhang, 2009) (G. Liu, 2013). But the related works based on SAR images are still limited. Due to the complex structure, it is hard to directly deal with the aircraft on the large image. A feasible way is that the detection algorithm is implemented firstly to generate ROIs which contain aircrafts, and then further processing will be carried out based the ROIs. In this procedure, aircraft segmentation is an important task, the base of feature extraction and target recognition. However, it is far from easy to segment the aircraft from the background completely and precisely in SAR images. Because of the complex structure, different kinds of electromagnetic scattering take place on the aircraft surfaces. As a result, aircraft targets usually appear to be inhomogeneous and disconnected in SAR images. It brings much trouble to target segmentation. Like CFAR algorithm (Novak, L.M., 1997), some classic methods, only utilizing image information, are able to extract strong scattering points of an aircraft according to intensity variations, but they fail to cluster the target and describe its shape accurately. Aircrafts in SAR images usually appears to be different from those in optical images. Complicated scattering mechanisms on aircraft surfaces, caused by the complex structure, usually lead to inhomogeneity of local intensities. In the process of image segmentation, some dark regions of the aircraft will be easily classified as the background. As a results, the aircraft will be segmented into several isolated regions. As conventional approach, morphological operations are carried out for clustering. But it may make a great influence on the original structure of the aircraft. The active contour model (ACM) is another successful segmentation method. Through minimization of the energy function, the active contour evolves towards boundaries. In SAR images, however, the multiplicative noise and intensity variations frustrate the evolution of the contour, and local minimums may be generated. In order to improve the segmentation performance, it is necessary to introduce the shape prior knowledge. In this point of view, the active shape model (ASM) is a valuable tool. ASM is firstly proposed by Cootes et al (T.F. Cootes, 1995), and it uses a statistical approach to model shape statistics of samples. A compact representation from a set of training instances is extracted in this method. Thanks to the help of shape information, the contour is controlled in the procedure of evolution. ASM has achieved success in meaningful object segmentation of medical and optical images. In the framework of ASM, dimensionality reduction is an important step in modelling shape statistics. Principal component analysis (PCA) has been widely used in many related works. This kind of linear learning techniques is usually based on a linearized shape space with limited deformation modes around a mean shape. It can keep good performance when shape samples in the training set are similar, otherwise it cannot embody the shape information of the samples fully. As for aircrafts, however, different types usually have different outlines. Two common types are taken as an example in Fig. 1 it can be seen that, the differences between (a) and (b) are obvious. When we implement ASM to deal with them, the training set should be consist of the shape samples of these aircrafts. Since PCA cannot extract full shape information of the training set when samples are not similar enough, the range of the contour deformation is limited, as shown in Fig. 2. This is not suitable for actual application. To solve this problem, a new aircraft segmentation method based on the non-linear shape statistics learning algorithm is proposed in this paper.

Describing a sample with Catmull-Rom Spline
Differentwith the traditional ASM, here we use CR spline to describe the shape samples, instead of the straight line, in order to decrease the control points and reduce computation. As for CR spline, presented by Catmull and Rom (E. Catmull, 1974), a curve segment can be defined by four control points at least. The control point is represented by . A curve segment can be defined by control points 0 1 2 3 , , , P P P P and knot sequence 0 1 2 3 , , , t t t t as follow  Yuksel, 2011), the centripetal CR spline has desirable advantages over the other types. Within a curve segment, it does not form self-intersections and cusps. Therefore, we use the centripetal CR spline to describe the aircraft shape samples. Since CR spline does not pass through the both end points, if we want the curve to go after 0 1 2 3 , , , P P P P , it is necessary to add 1  P and 4 P to the ends respectively. They make a less influence on the cure, and can be define as . With this setting, the shortcoming can be rectified. Due to our limited SAR data capacity of aircraft targets and the lack of ground truth, which are the common problems of many other researchers, the shape training set cannot be established based on SAR images. To solve this problem, a series of vertical views of aircrafts are collected from the Internet. We build the shape samples according to these images. As a result, 69 samples are obtained, in which 42 ones belong to Type A and 27 ones belong to Type B. The typical samples are shown in Fig.3. The shape model is consist of 23 control points that represent the stationary local features respectively. Since the training set should contain the shape information only, In order to build a shape space, where there is only the shape difference among samples, it is necessary to remove the other differences including translation, scale and rotation. That is why all the samples in the training set should be normalized and aligned. Cootes' method can be used for reference.  To achieve the shape alignment between shapes 1 k  x and k x , a modified Procrustes algorithm is provided (J. C. Gower, 1975).
The scheme of training set alignment is as follow a) Align every shape with the first shape. b) Obtain the mean shape of the aligned shapes. c) Realign the shapes with this mean shape. d) Repeat (b) and (c) until the process converges.

Dimensionality reduction by ISOMAP
Isometric feature mapping (ISOMAP) is the extension to multidimensional scaling (MDS), a classic method for embedding dissimilarity information into Euclidean space. ISOMAP returns an embedding, where the distance between points is approximately equal to the shortest path distance. And it obtains the estimate of the dimensionality of the underlying manifold. The specific steps are as follows. a) Compute the 2 L distance between all pairs of points b) Define the set of points comprising the neighbourhood by k-nearest neighbours (KNN). c) Define a graph with a node for each input point and weighted undirected edges connecting each node to the nodes corresponding to the points. d) Solve for all-pairs shortest paths on this sparse graph to calculate a complete pair-wise distance matrix. e) Use MDS to find an embedding of these distances in Euclidean space. The shape prior estimation based on ISOMAP is described as follows. a) Project the training set shape samples x is given by where the normalization parameter  is set in the experiments. The shape in the high-dimensional space is computed using high-dimensional features of the k nearest neighbours and the reconstructed weights (T.F. Cootes, 1995). Therefore, the estimated shape prior is

IMGAE MOTIVATION BY GVF SNAKE
In the proposed method, the image data motivation is realized by snake model based on edges.

Edge extraction by ROEWA operator
The gradient operator used in traditional snake and GAC models is the conventional edge detector. It does not work well in SAR images because of the multiplicative nature of speckle noise. Different from the preceding works, the ROEWA operator, an optimal multi-edge detector for SAR images (Fjørtoft R., 1998), is employed to reduce the influence of speckle. Although ROEWA operator can generate the edge strength satisfactorily, it cannot get the correct edge direction, which brings trouble to edge location and thinning. The watershed algorithm can give closed boundaries, but it easily leads to the over-segmentation. Though this phenomenon can be dealt with by some variant method (Sheng, G., 2012), manual markers are required.
To extract edges from the edge strength map, the multi-direction NMS is applied. In the absence of edge direction, the NMS is used in 8 different orientations (0, 45°, 90°, 135°, 180°, 225°, 270°, and 315°) to find the local maximas. The result is further filtered by a two level thresholding. The high threshold HTh and low threshold LTh are set as follow: the ratio of the number of non-edge pixel to the total amount of pixel , besides the pixel borders on strong edges, it is classified as the weak edge. Otherwise, the pixel does not belong to edges. Finally, a size filter is applied to eliminate isolated pixels.

GVF snake
A snake contour is an elastic curve that deforms to minimize the following energy functional.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium "Developments, Technologies and Applications in Remote Sensing", 7-10 May, Beijing, China where the internal energy int E derives from the contour itself and the external energy ext E derives from the image. Since the contour evolution is time-consuming, especially when the contour is long, the spline curve is introduced to form an economical realization of snake. The internal energy is not required in the spline-based snake because of the restrictions of the spline. The B-spline snake, also called B-snake, has been widely used. In this paper, we choose the CR spline, which is easy to build the shape models.
On the other hand, since the capture range of the traditional snake is very limited, Xu and Prince improved the external force by introducing the gradient vector flow (GVF) field ( , ) [ ( , ), ( , )] V x y u x y v x y  (Xu C., 1997), which minimizes the energy function where  is a regularization parameter, and f is the edge map of the raw image.

METHODOLOGY
We want to move the points of the shape to the edges of the image while still satisfying the shape constraint of the model.
Firstly the mean shape x  is transformed into X  by the parameters ,, s  t . Then X  is motivated by the GVF field V , and the deformed shape is GVF X . Then the new shape new X can be obtained according to GVF X , via shape prior estimation based on ISOMAP. During this process, the parameters is updated and the new ones ,, s     t are equal to the values that align x  to GVF X . The above operations are repeated, until the contour becomes stabilized. The contour evolves during iterations and always keeps similar to the shapes of the training set during the deformation. In the first iteration, the parameters should initialized.

EXPERIMENTS
To demonstrate the performance of the proposed segmentation method, it is compared with two ACMs including RSF and LGIF. The segmentation results are presented in Fig. 4. The aircraft chips are picked from the Ku-band airborne SAR images with 0.5 m resolution. Since the aircrafts usually appear to be inhomogeneous in SAR images, RSF and LGIF models only extract some isolated bright pixels and fail to find the real contours. It can be seen that the performance of them is unacceptable. By contrast, the proposed method extracts the complete contours and recovers the real shapes using the PDM. It is much better than the other two methods. What's more, the proposed method can deal with the two types properly. In this framework, when we need to deal with other types of aircraft, just add the corresponding samples to the training set. The predictability is much improved.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium "Developments, Technologies and Applications in Remote Sensing", 7-10 May, Beijing, China

CONCLUSION
A refined segmentation method for aircraft target in SAR images based on ISOMAP is proposed in this paper. It combines the image information and the shape prior to improve the segmentation accuracy. On the one hand, the image motivation is carried out by the GVF snake. On the other hand, the shape information is extracted by ISOMAP algorithm. The experiment based on real SAR data shows that the proposed method achieves obvious improvement in accuracy.