MORPHOLOGICAL MOMENTS OF BINARY IMAGES

The concept of morphological moments of binary images is introduced. Morphological moments can be used as a shape descriptor combining an integral width description of an object with a description of its spatial distribution. The relationship between the proposed descriptor and the disc cover of the figure is discussed and an exact analytical method for descriptor calculation is proposed within the continuous morphology framework. The approach is based on the approximation of the shape by a polygonal figure and the extraction of its medial representation in the form of the continuous skeleton and the radial function. The proposed method for calculation of morphological moments achieves high accuracy and it is computationally efficient. Experimentations have been conducted. Obtained results indicate that the morphological moments are a more informative and rich shape descriptor than the area of the disc cover. Application of morphological moments to the font recognition task improves the recognition quality.


INTRODUCTION
Pattern spectrum introduced in (Maragos, 1989) is one of the common techniques of morphological analysis of images.It describes the contribution of primitives of different sizes to the image formation, and in the case of binary images and the choice of a disc as a structuring element, it can be seen as an integral description of the width of objects in the image.One can consider as a disadvantage of pattern spectrum its inability to capture the spatial information contained in the image.In particular, if a binary image contains several objects represented by connected components, then the spectrum does not depend on their mutual arrangement, unless they merge with each other.To avoid this drawback, several modifications of the pattern spectrum were proposed.Generalized pattern spectrum (Wilkinson, 2002) calculates not the area (the sum of gray values) of the difference between the results of two successive morphological operations, but the cumulative value of the power function of its coordinates; the spatial size distributions (Ayala and Domingo, 2001) analyze the difference between the geometric covariograms for binary images or the auto-correlation function for gray-scale images of the original image and its granulometric transformation; multi-scale connectivity (Braga-Neto and Goutsias, 2005) simulates changes in the connectivity of objects on the image when zooming; sizedensity spectra (Zingman et al., 2007) uses a less rigid version of the opening operation, when it is sufficient that the proportion of the area of the primitive overlapping the image is greater than a certain density value.In all these modifications traditional pattern spectrum is a special case of more complicated technique.
It is easy to see that the generalized pattern spectrum is closely related to the well-known concept of image moments.Statistical moments are applicable to many different aspects of image analysis ranging from invariant pattern recognition and image encoding to pose determination.When applied to images, they describe the image content (or distribution) with respect to its axes.They are designed to capture both global and detailed geometric information about the image.As an alternative to classical geometric moments the moments of Zernike can serve (Khotanzad and Hong, 1990), which are based on the orthogonal system of polynomials, so that the moments are independent.Despite the fact that moments can be used in image reconstruction (Xin et al., 2005), the features obtained by moments are not sufficiently informative, since the moment of a given order is just a scalar value.
The aim of this paper is to combine the advantages of pattern spectrum and image moments and get a shape descriptor that reflects both structural and spatial features of the form.The idea is to calculate the moments not only for the images themselves, but also for the results of morphological operations applied to them.This descriptor is described in terms of continuous mathematical morphology, which will ensure the high efficiency of the procedure for its calculation.
The rest of the paper is organized as follows.In Section 2 theoretical concepts of morphological moments are given.We focus on the case of the disc structuring element and give the definitions of the moments in terms of the thickness map for the discrete case and in terms of the disc cover -for the continuous one.Section 3 is devoted to the description of an exact analytic algorithm for a class of polygonal figures.Finally, in Section 4 we show the usefulness of our descriptor in the problem of font recognition and also estimate the time costs for calculating the moments.

Pattern Spectrum and Image Moments
The original idea of pattern spectrum proposed by Maragos (Maragos, 1989) is based on Serra's Mathematical morphology filters (opening/closing).More formally, let X be the given binary image (pattern).Let B be the structuring element centered the origin on the 2D object plane P .The parametrically scalable structuring element B(r) cab be defined as B(r) = {rb|b ∈ B}, r ≥ 0, b = (x b , y b ) ∈ P Let X ⊂ P and B ⊂ P , then the morphological pattern spectrum (PS) of X is defined as where S(X) = |X| L 1 is the area of X, S(X • B(r)) is the area opening and S(X • B(r)) is the area closing of a set X by a strucuring element B(r).
P S(r) is the spectrum for positive part of the axis r (spectrum of image objects), P S(−r) is the spectrum for negative part of the axis r (spectrum of image background).This means that S(X • B(r)) is a quantitative measure of B(r) in X.Hence, the pattern spectrum is defined as a morphological tool that gives the quantitative information about the shape and sizes of the objects in the image.The size distribution is represented in the form of histogram for further processing.
Since it is inconvenient to carry out computations with derivatives, in practice a discrete morphological spectrum of continuous image is used: (2) where ri = i∆r, i ∈ Z, ∆r is the scale step.
Simple geometric properties of an image such as area, position, and orientation can be easily computed from a set of linear functionals of the image called geometric moments.Hence let f : Ω ∈ R 2 → R, Ω being some compact set, be an image function describing a real scene, such that 0 ≤ f (x, y) represents an intensity of the image at a spatial position (x, y) ∈ Ω, where Ω is often called the image plane.
We define the (p, q)-th of (x, y) as follows mpq = Ω x p y q f (x, y)dxdy. (3) If an analog original image function f (x, y) is digitized into its discrete version {f (xi, yj)} with an W × H array of pixels, the double integration of 3 must be approximated by summation.
Here (xi, yi) is the centre point of the (i, j).A commonly used prescription to compute mpq from a digital image is defined as where ∆ = xi − xi−1 = yj − yj−i is the sampling interval.
A number of fast algorithms and hardware implementations for determining mpq have been proposed (Dai et al., 1992), (Flusser, 1998).It is clear, however, that mpq is not a very accurate estimate of mpq, particularly when the moment order (p, q) increases.
Further, we pay special attention to the case of binary input datasome two-dimensional function, which takes zero or one at every point.A discrete function example is a binary image.A continuous analogue is a scene describing a set of figures, where figures are closed regions on the plane bounded by a finite number of disjoint closed Jordan curves.This function takes the value of 1 if the point belongs to some figure, otherwise the function takes the value of 0. The moment of the figure X from the continuous case is given by the following equation: x p y q dx dy. (5) We combine two methods described above.For this purpose, we join their parameters' sets and define a function, which depends both on the non-negative integer moment order (p, q) and on the real-valued radius r.
Definition 1.The morphological moment of the order (p, q) of the figure X is a function that describes the dependence of the moment on the size of the chosen primitive.The moment is calculated by the opening of figure with a circle of radius r: In this case conventional moments are defined as Mpq(0) and the positive part of the pattern spectrum is given by the following equation: To calculate the moment for the discrete case we use the formula: The x p y q . (8) Note that equation ( 9) gives us the generalized pattern spectra (Wilkinson, 2002).
An illustration of the introduced concepts is shown in Fig. 1-2.

Morphological Moment Invariants
The morphological moments' normalization is important to ensure invariance to shift, rotation and scale changes.The necessary invariants (Hu, 1962) are obtained for traditional moments.Note that conventional central moments of the first order and the zero-order scale invariant are equal to the constants (0 and 1, respectively) and carry no information about the object.This is because we can not use the area and the position of the object in problems of invariant recognition.
In the case of morphological moments the situation is somewhat different since the moment is a function of the width parameter r.Even if we can not use the information about the area and the center of mass, the information about their changes, when the radius r of opening is growing, is extremely useful.Therefore, it makes sense to fix the center of mass of the entire figure and to make shifts according to this value by analogy with central moments: Invariants ηij with respect to both translation and scale can be constructed from central moments by correcting both the argument and the value of the moment through a properly scaled zeroth central moment at the point 0: We can assume that, as a result of normalization procedure we get the moments of the figure, which is obtained from the original figure by the area normalization and by the center mass translation to the origin without rotation.
Rotation invariance is achieved by means of the transformations that are used for the Hu invariants.The only difference is that we perform operations on functions and not on scalar values.

Moments Computation Based on Thickness Map
Obviously, the question of Mpq(r) calculation for a given r arises.A naive approach requires the result of morphological opening X • B(r) in the binary image form explicitly.While the algorithms for computing basic morphological filters are well developed, the necessity for a discrete model does not allow to obtain a method that could be used in real-time systems.
In the paper (Sidyakin, 2013), the new definition for pattern spectrum with disc structuring element was given based on disc thickness maps.Let F be a binary figure which fully fits on the frame K : F ⊂ K. Denote F C(K) = K\F , the background of figure F on the frame K. Then the binary image, that correspond to the figure F , is defined as: Let B(q, r) be a translatable flat disc with the center at the point q = (x, y) = B(q, 0) and the scale parameter r.Definition 2. Thickness map TB(fF ) of a binary image fF (x, y) with disc structuring element B(q, r) is real-valued image defined on the frame K, with each point representing the maximum size of its covering disc structuring element fully inscribed in the figure shape F. For the background of figure F , the value of the scale parameter is negative.
In particular, it was also proven in (Sidyakin, 2013) that discrete Maragos pattern spectrum with disc structuring element is a histogram of discrete disc thickness map.The proposed disc thickness map made possible the creation of precise fast disc pattern spectrum computation algorithm.
It is easy to show that the point (x, y) belongs to the opening with radius r if and only if TB(x, y) ≥ r.This allows us to calculate the morphological moments on the basis of the thickness map: To obtain the thickness map one can use the algorithm described in the paper mentioned above.This algorithm extracts the continuous skeleton of the binary shape and then surprisingly rasterize it.As a result, circles with centers at discrete skeleton points are considered for thickness map computation.That is why the algorithm (Sidyakin, 2013) can be considered discrete-continuous.This approach allows to significantly reduce the time of the disc spectrum calculation and the moments' calculation in comparison with the fully discrete approach.
However, a completely continuous approach to the calculation of morphological moments is possible, which we describe in the next section.

CONTINUOUS ALGORITHM FOR MOMENTS CALCULATION
3.1 Disc cover of a polygonal shape Methods of binary object approximation are well-developed and widely used.One of the most remarkable continuous shape models is a polygonal figure (a polygon with polygonal holes).Using polygonal shapes, it is possible to approximate the boundaries of complex objects, represented by nonlinear curves or discrete bitmap images, with high accuracy.The other advantage is the availability of highly efficient shape processing algorithms developed on the basis of computational geometry.A method for calculation of analytical integral description of the polygonal figure width is presented in the paper (Lomov and Mestetskiy, 2017) and it is based on the figure disc cover.The disc cover is a set of inscribed discs of a given size.The key role in this work is played by the notion of medial representation, which includes a continuous skeleton (the set of centers of circles inscribed in the figure) and a radial function that corresponds to the radius of the inscribed circle at each point of the skeleton.
The algorithm consists of the following steps: The third step is of special importance for us in this paper.In the aforementioned paper, it is shown that the disc r-cover coincides with the figure opening performed with disc of radius r.The disc r-cover is a geometric figure (or a set of figures) bounded by straight lines and convex circle arcs.During the work of the algorithm, disc cover gets an explicit representation in the form of the grouping and elimination of simple geometric shapes (Fig. 4), among which are: • Bicircles' proper regions (shown in light grey) • External sectors of truncated bicircles' small end circles (dark grey) • Lenses in intersections of the external sectors (red) The bicircle is obtained from its proper region by adding the outer sectors of the end circles.An proper region of any type is a polygon, each vertex of which is either the end of the bicircle's axis or its projection onto the generating site (in the case of the point site, the projection coincides with the point itself).Therefore, any proper region can be represented as a union of triangles.As required by the algorithm (Lomov and Mestetskiy, 2017), the non-monotonic bicircles split into monotonic pairs before moments' calculation .All bicircles undergo corrections when radius r increases -circles with an insufficiently large radius are removed.Note that as a result of these operations we obtain bicircles whose proper regions are also polygons.
The end circles' sectors are circular sectors.Finally, the lens can be divided into two parts, each of which represents the complement of a triangle to a circular sector 6.
Thus, the problem can be reduced to the calculation of moments of regions of two types: triangles and circle sectors.

Moments of Primitive Regions
An arbitrary triangle can be represented as the result of unions and complements of triangles with two sides parallel to the coordinate axes (primitive triangles).Let A, B, C denote the triangle Figure 7. Representation of the triangle using triangles with sides parallel to the coordinate axes points.These points are ordered in increasing order along the abscissa.If the point B is not a medium point along the ordinate, we are dealing with the partition of type 7a, otherwisewith the partition of type 7b.Therefore, the triangle ABC can be represented as in the first case and as in the second case, where cl is a closure of a set.
In this representation sets that are joined intersect in a set of zero measure, and the second set is embedded into the first when taking the set difference.Therefore, for the moments of triangle ABC one of two following equalities is justified: A similar equation can be composed for any other triangle.
Primitive triangle coordinates can be denoted as (x0, y0), (x0, y1), (x1, y0), so the angle at the vertex (x0, y0) will be direct.Then the integral for calculating the moment can be represented as: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W4, 2017 2nd International ISPRS Workshop on PSBB, 15-17 May 2017, Moscow, Russia (x 0 , y 0 ) At each stage of integration, the integrand is a polynomial in one variable, so that the integral is easily calculated.
The integral on a circular sector of the function f (x, y) = x p y q is easier to consider in polar coordinates.In this case, the integral can be written as: Expanding the brackets, we get under the integral the sum of monomials of the form a r b+c+1 cos b φ sin c φ. Repeatedly using the product-to-sum identities for trigonometric functions: we can convert the trigonometric part of monomial cos b φ c φ to the form of k i=1 aigi(niφ), where gi(x) is sin(x) or cos(x), and ni ∈ Z.Further evaluation of the integral is not a problem.
As a result, the moment of polygonal figure cover can be found as the sum of the moments of proper areas of bicircles -full and truncated -and the moments of small end sectors of truncated bicircles minus moments of lenses in the intersection of these sectors.All relevant moments are calculated analytically.

EXPERIMENTS
The method based on the area of disc cover has proved itself well for solving the task of recognizing computer fonts in a certain context.Experience has shown that a significant part of the errors in font recognition resulted from the non-distinction between the straight and italic typefaces.Indeed, as shown in the upper figure (Fig. 9), the symbol width features often does not vary when typeface is changed from straight to italic.However, these symbols can be distinguished by the moments of higher orders (the two lower figures).
The purpose of the experiment described below is the evaluation of improvement possibilities of the previous method by using not only the zero-th order moment (area) but also higher-order moments.
The experiments are conducted on the same database containing images of 52 Latin letters (26 lowercase and 26 capital letters) from 1, 848 font typefaces taken from the collection of digital fonts owned by Paratype Company.For the reference images, diagrams of morphological moments are obtained by the method proposed above.For this purpose each symbol is drawn as a binary raster image at such a scale that the height H of the capital letter is 1000 pixels.Continuous skeletons are constructed from these images using the method described in (Mestetskiy and Semenov, 2008).Morphological moments up to the order of 3 with radius step 0.5 of the pixel value are calculated based on continuous skeletons.
For the same fonts, images of symbols in a lower resolution are obtained so that the height H of the letter is 100 and 50 pixels.
The moment diagrams are also constructed for these symbols.The radius step in the calculation process is 0.05 and 0.01 pixel, respectively.These diagrams are normalized in such a way that they could be compared to diagrams of standard font symbols.
The normalization consists of a stretching by 100 times in the ordinate direction and 10 times in the abscissa direction and 400 times in the ordinate direction and 20 times in the abscissa direction for low resolutions 100 and 50, respectively.As a result, all normalized diagrams use the same set of radii values.
To compare the time costs with the fully discrete method (implementation based on the OpenCV library) and the discretecontinuous method (based on the algorithm (Sidyakin, 2013) Creating of the skeletons and the calculation of moment diagrams (10 moments for 52 glyphs of 1848 fonts) took in total about 8 hours on the computer with Intel Core i5 processor and 6GB of RAM.
Further, for each font images of the 1000 common English words (the average length of the word is 5.33 characters, in the set there are very short words, for example, I, be), random 40% of which are converted to upper case, are composed from the letters in low resolution.These images are used as the test set.Next, the moment diagrams of the letters on test images are compared with the diagrams of reference images in L1 metric.As an integral font similarity metric we use a linear combination of distances between all characters present in the word.The coefficients of the linear form for each word are obtained by training on the entire set of test fonts.In the experiment, we calculate the distances for 52 letters between all pairs of 1848 typefaces, which take 54 minutes.This means that the time of the request -checking the typeface in the basis of the references -is 1.75 seconds.
The experimentation results show that the average level of correct font recognition by word when using only the zero-th order moment is 91% at scale 100 and 69% at scale 50, adding the central moments of the first order increases the recognition accuracy to 95% and 87%, respectively.
Thus, the conducted experiment confirms the efficiency of the proposed method and shows its effectiveness in the practical task of comparing a large number of images (1848 × 1848 × 52) with quite high recognition quality.

CONCLUSION
The proposed descriptor and the method of its calculation develop the possibilities for applying highly effective algorithms of computational geometry to the analysis and recognition of the image shapes.Known discrete approaches to the calculation of shape width descriptors have a high computational complexity.
The proposed continuous model of morphological processing designed for polygonal figures on the basis of a disc cover make it possible to decompose the original problem and reduce the computations to taking simple integrals.
The developed algorithm is the first approach that allows to obtain an exact analytical representation of the spatial distribution function of the polygonal shape width.The approximation of raster objects with polygonal figures provides an opportunity to use the method for image recognition and analysis.The high efficiency of the proposed method allows for real-time comparison and measurement of shape width similarity in space.

Figure 1 .
Figure 1.Morphological openings of the lizard figure.The sizes of the structuring elements are depicted on the right.The center of mass is marked with red.

Figure 2 .
Figure 2. Graphs for the moments of different orders are given for the lizard figure.

Figure 3 .
Figure 3. Continuous model of disc cover of the binary image.The binary image (a) is approximated by the polygonal figure, for which the medial representation (c) is constructed, which describes the disc cover (d). 1. Obtain the medial representation of the polygonal figure in the form of continuous skeleton and radial function on the basis of the Voronoi diagram of linear segments.Linear segments form the polygonal figure boundary.2. Obtain the representation of the complex polygonal figure in the form of the union of elementary geometric figures, called bicircles.The bicircle is a union of the circles inscribed in the figure with the centers on the edge of the skeleton.3. Represent the figure disc cover as a union of bicircles' subset and calculate the area of the disc cover by means of bicircles' areas.4. Construct the distribution of the disc cover area as a function of the disc size.

Figure 4 .
Figure 4. Disc cover components of the polygonal shape.The cover area consists of the sum of the areas of proper regions and sectors minus the sum of the areas of the lenses.But nothing prevents us from preserving the general algorithm structure.Therefore, we replace the calculation of the areas of the regions composing the figure by the calculation of their moments on the third step.To do this, consider the regions of each type in more details.

Figure 6 .
Figure 6.Representing lenses through triangles and circular sectors

Figure 9 .
Figure 9.The straight and italic symbol typeface (a).Graphs of zero-th and first order moments (b); Blue -normal, reditalic.
), time is measured for symbol processing of 20 randomly selected fonts at different scales.The step is chosen to be equal to 1, since discrete and discrete-continuous methods allow to use only integer values of radii.The results (Tab. 1) demonstrate the speed superiority of the proposed method over discrete and discretecontinuous methods, especially for large-scale images.