COMPARATIVE EVALUATION OF PAVEMENT CRACK DETECTION USING KERNEL-BASED TECHNIQUES IN ASPHALT ROAD SURFACES

With the increasing demand for the digital survey and acquisition of road pavement conditions, there is also the parallel growing need for the development of automated techniques for the analysis and evaluation of the actual road conditions. This is due in part to the resulting large volumes of road pavement data captured through digital surveys, and also to the requirements for rapid data processing and evaluations. In this study, the Canon 5D Mark II RGB camera with a resolution of 21 megapixels is used for the road pavement condition mapping. Even though many imaging and mapping sensors are available, the development of automated pavement distress detection, recognition and extraction systems for pavement condition is still a challenge. In order to detect and extract pavement cracks, a comparative evaluation of kernel-based segmentation methods comprising line filtering (LF), local binary pattern (LBP) and high-pass filtering (HPF) is carried out. While the LF and LBP methods are based on the principle of rotationinvariance for pattern matching, the HPF applies the same principle for filtering, but with a rotational invariant matrix. With respect to the processing speeds, HPF is fastest due to the fact that it is based on a single kernel, as compared to LF and LBP which are based on several kernels. Experiments with 20 sample images which contain linear, block and alligator cracks are carried out. On an average a completeness of distress extraction with values of 81.2%, 76.2% and 81.1% have been found for LF, HPF and LBP respectively.


INTRODUCTION
Asphalt road pavements are subject to degradation due to continuous traffic loading, "frost heaving", "joint failure", weather conditions and also lack of timely maintenance and repair.Failures in asphalt pavement which are characterized by cracks, potholes, patches and ruts, lead to significant geometric failure and the consequent disruption of motorability.Asphalt pavement cracks can be categorized into fatigue (alligator), longitudinal, transverse, block, slippage, reflective and edge cracks (Orr, 2006).Cracks mostly show up as linear structures in images.Extraction of those structures often employs feature extraction processes.One of the major challenges in feature extraction is dealing with varying scales and orientations.Feature extraction methods that can provide scale-and rotation-invariant feature representations are important in pattern recognition.In this research, three kernel-based segmentation methods are used and compared for the automated detection of cracks in RGB images.The compared methods are: (i) line filtering (LF), (ii) high-pass filter (HPF) and (iii) local binary pattern (LBP).For HPF and LBP filters rotation invariant versions exist.To obtain rotation invariance for LF, a sequence of rotated filter kernels can be applied.By processing the images on different image resolution levels multiple scales are taken into account.In the experimental investigations the research is concentrated on the detection of cracks and the completeness of the detected cracks.Many studies have been carried out to detect the cracks from mobile recorded images using different algorithms.* Corresponding author Gavilán et. al. (2011) proposed crack detection following a seed-based approach.Initially local minima of a grey scale image (in the paper detected by multiple directional nonminimum suppression) are selected as seed points.In further steps routes between the seed points are attained using path growing techniques.(Mancini, et al., 2013) detected pavement cracks by applying a Gradient Vector Flow (GVF) snake proposed by (Xu & Prince, 1997).An edge map calculated by gradients of the scene is introduced into a variational formulation of a vector field that minimizes an energy functional.They report about successful applications of GVF snakes on a variety of different cracks.Wei et al. (2010) detected the pavement cracks from images based on a beamlet transform.Input to the beamlet transform is a binary image of detected cracks which is generated by segmentation of the original image.Wei et al. (2010) (Oliveira, 2013).In the second method, first the images are segmented based on an automatic histogram thresholding.The crack candidates are grouped by a connected components algorithm if they fulfil some geometric requirements.The remaining ones are linked and the linked connected components are considered as 'global' cracks (Oliveira, 2013).
The rest of this paper is organized as follows.Section 2 explains the methods used for pavement crack detection in the current study.The study results and discussions are presented in section 3. Section 4 reports about the comparison of crack detection using LF, HPF and LBP.The final chapter closes with summary and outlook.

METHODS
Cracks appear mostly as dark linear structures or narrow and expanded dark regions in images.Moisture in the cracks intensifies the dark impression of the cracks.A high image resolution with e.g. a ground element size of 1 or 2 mm enables that even very narrow cracks are visible in the images.In the next three subsections the filtering techniques used in this study are outlined.

Line filtering
Line filters may range from very small windows sizes of 3× 3 pixels to larger masks of for example 21× 21 pixels.Eq. 1 shows the structure of a vertically oriented line filter mask .A convolution with this mask averages the neighbourhood left and right from the central pixel and subtracts the central pixel value from the average.The width of e.g. 10 pixels left and right from the centre should guarantee that sufficient neighbourhood is taken into account.The height of the mask serves for averaging in the vertical direction.With the height of the mask we probe the length of straight line segments.From a theoretical point of view width and height of the filter can be different.Our experimental investigations indicate that a similar extend in both directions seems to be appropriate.
The '*' in Eq. 1 indicates the convolution of the horizontal and vertical kernels.To take many directions into account the mask (Eq. 1) is incrementally rotated with rotation angles of 10° to generate 18 quadratic line filter masks   (Miraliakbari et. al., 2014).Thus an image has to be convolved with these 18 filters rather than with one.
For the detection of cracks the image  is represented on different resolution levels .

High-pass filter
For high-pass filtering of the image on all resolution levels ′  =   *  a Laplacian kernel is used with Instead of smoothing the image with a Gaussian we prefer to smooth the image with a bilateral filter according to (Paris, et al., 2009) (Gonzalez &Woods, 2002).
The Laplacian filtered image is thresholded by histogram analysis to detect the dark lines or (small) regions which are the localized potential crack locations on image resolution level k.Fusion over all resolution levels is carried out in the same was as in the LF approach.

Local binary pattern
LBP is proposed by (Ojala, et al., 2002) and applied by (Hu & Zhao, 2010) for crack detection.LBP is a multiresolution approach for texture classification which is grey-scale and rotation invariant.In LBP the neighbourhood of  equally spaced pixels on a circle of Radius  is used to define texture as illustrated in Figure 1.
Figure 1.'Circularly symmetric neighbour sets' for P=8 and R=1 (adopted from Ojala, et al., 2002) Texture  is introduced according to where   is the grey level of neighbouring pixel ,   is the grey level of the central pixel  ,  is the texture operator and Eq. 5 makes the operator grey-scale invariant.The operator  is developed in such a way that it addresses all local binary configurations (the 36 possible rotation invariant binary patterns) visualized in Figure 2.
According to (Ojala, et al., 2002), the patterns in the first row of the figure are called "uniform" patterns and have values between 0 to 8 while value 9 is assigned to the rest of the patterns.
Figure 2. The 36 possible rotation invariant binary patterns (Ojala, et al., 2002) For detecting cracks we consider only the patterns marked with numbers 3, 4 and 5 into account.If a pattern 3 or 4 or 5 is detected at a certain pixel this pixel is considered as a potential crack pixel.Fusion of the detected patterns is carried out by taking the operation.The result is  3,4,5  .
The texture operator is highly discriminative because of Eq. 5 thus the detected potential cracks are noisy.If grey-scale invariance is not required (Ojala, et al., 2002) have to be fused over all resolution levels in the same as in the other two approaches.

Post-processing
In this step small remaining regions which are most probably non-cracks are removed using the geometric properties area () and roundness () of the image regions.Roundness is defined by  = 4/  2 where  is the perimeter of the region.Small regions and regions with relatively high roundness are removed with the effect that a large amount of non-crack objects are eliminated.Thresholds for the area are found by histogram analysis.The next step is to remove road mark borders.Road marks appear bright in the images and often have a dark border due to aging, shadow or moisture.This knowledge is used to eliminate the corresponding potential crack elements.
In LBP method some further morphological processing is carried out e.g.closing is utilized to fill the empty space between the detected crack edges.

RESULTS
In the experiments 20 sample images are investigated which contain linear, block and alligator cracks.The images have a ground resolution of approximately 1 mm to 1.5 mm.For ground truthing cracks are detected manually in the images.The images are taken at various weather conditions and illuminations.Figure 3 shows 12 out of the 20 image data.
Figure 3.Some image data containing linear, block and alligator cracks The matching for all variants (LF, HPF and LBP) is carried out on the resolution levels which vary from the coarsest level (1/8) and ascends to the highest image resolution (1/1).The downsampling enables detection of cracks with different widths.
Figure 4 shows the detected crack pixels using LF variant on the four different resolutions.The detected crack pixels on level 4 (coarsest level) looks quite proper but on the other hand not all cracks (in particular the narrow cracks) have been detected.Potential detected crack pixels on level 1 (finest level) are quite noisy.The main reason of the noisiness in level 1 is that all tiny borders between asphalt and aggregates have been detected as potential crack pixels.
Figure 5 shows the union of the potential crack pixels results on all levels.As shown in this figure, both narrow and wide cracks are represented.Additionally we obtain noisy results together with some artefacts (non-crack regions) which must be removed in the post-processing step.This is also true for the other two variants.Figure 6 presents a sample of post-processing results applied on the potential crack image generated by the LF variant.The original image contains road asphalt, aggregates, cracks, a dry leaf and portion of a road mark.The next image is the raw result of crack detection containing cracks, noise and non-crack regions.Once small areas are filtered, the results look clean (third image).Furthermore using roundness (fourth image from top), all regions which have roundness higher than a certain threshold (in this example threshold amounts to 0.12) will disappear.Finally the border between the road mark and the asphalt is eliminated using image histogram analysis.

COMPARISON OF CRACK DETECTION USING LF, HPF AND LBP
Apart from any pre-and post-processing applied for these variants, HPF is the fastest method of crack detection compared to the other two variants.The main reason is that only one template is used and the kernel itself is rotation invariant.LF uses 18 kernels in our implementation is therefore computationally more expensive.Remarkable (even though expected) is that in LF edges which stem from the border between shadow and sun areas never showed up in the extracted line structures.LBP acts as edge detector due to the nature of the selected LBP patterns.Due to the high sensitivity of LBP, edges caused (mostly) by bright aggregates which are in the vicinity of the crack area are connected to the crack pixels.This connection effect is more visible on the finest resolution level.Therefore further morphological operations are used for LBP to disjoint the non-crack pixels from the crack region (erosion) and fill the empty space between crack edges (closing).Figure 7 shows final crack detection results using LF, HPF and LBP.
For all 20 samples the achieved completeness of crack detection is shown in Table 1 with respect the LF, HPF and LBP.In post processing 'optimal' roundness values of 0.2, 0.5 and 0.2 for LF, HPF and LBP have been applied for all samples.LF has a relatively overall sensitivity to the linear structures in all samples; in none of the experiments the completeness was less than 60%.With HFP three of the 20 samples have a completeness of less than 50%.For LBP one of the completeness results is below 40% which is the worst of all results.
Image Nr.The graph reveals that a further increase of completeness can be achieved by fusing the detection results of all three filtering approaches.

SUMMARY AND OUTLOOK
Cracks are one of the critical distress features in road condition mapping and evaluation.The comparative evaluation of kernelbased segmentation methods for detecting cracks in images in this study is focused on line filtering, local binary pattern processing and high-pass filtering applied to images which are represented on several resolution levels.Depending on the filter different pre-and post-processes are applied.For HPF, for example, a bilateral filtering is used to reduce the noise sensitivity of the filter by smoothing the images while preserving edges.For LBP, morphological filtering disjoints the non-crack pixels from the crack region.For all three variants noisy small line and edge segments are eliminated with respect to the expectation that crack regions should have a minimum area size and certain non-roundness.
The comparison of the detection results of 20 different images obtained by the three filtering variants indicates that LF shows the highest overall completeness of detected cracks.If cracks are hardly visible in an image the high sensitivity of LBP filter has some advantages but mostly the high sensitivity is more a problem than an advantage.Due to the rotation invariance of the HPF filter this variant offers the computationally least expensive option.The LBP performed better than HPF with respect to the overall completeness but the only detection result observed in the experiments with a completeness of less than 40% stems from the LBP.
A further increase of completeness can be achieved by fusing the detection results of all three filtering approaches.

Figure 5 .
Figure 5. Original image (top), discovered potential crack pixels by LF, (union on all levels of detail, bottom) Due to the high sensitivity of LBP, edges caused (mostly) by bright aggregates which are in the vicinity of the crack area are connected to the crack pixels.Figure6presents a sample of post-processing results applied on the potential crack image generated by the LF variant.The original image contains road asphalt, aggregates, cracks, a dry leaf and portion of a road mark.The next image is the raw result of crack detection containing cracks, noise and non-crack regions.Once small areas are filtered, the results look clean (third image).Furthermore using roundness (fourth image from top), all regions which have roundness higher than a certain threshold (in this example threshold amounts to 0.12) will disappear.Finally the border between the road mark and the asphalt is eliminated using image histogram analysis.

Figure 6 .
Figure 6.From top to bottom: original image, raw detection results, after filtering small areas, after filtering by roundness (0.12), after filtering boundary of road marks

Figure 7 .
Figure 7.Comparison of crack detection using three different methods; original image (top left), final result of crack detection using LF (top right), HPF (lower left) and LBP (lower right) For validation of the crack detection we focus on completeness:  =      +

Figure 8 .
Figure 8. Percentage of crack detection completeness regarding to LF, HPF and LBP The convolution of   with all masks   ′ , =   *   (2) is identical with the correlation   ⋆   because the masks   are symmetric with respect to the centre point.To scale the correlation result to the normalized cross correlation of image window and mask, ′ , is normalized by the product of the standard deviations of the image window    and the mask   and by the size of the window.The normalized result  ′ , /(      2 ) is denoted by  , .  >   =   .Fusion over all resolution levels is carried out by merging the detection results (  ∀).
On every resolution level , each  , contains  different correlation results related to the different orientations of the masks.By taking the ( , ∀) the highest correlation coefficient   on level k is found.Potential crack locations on level k are localized by thresholding the correlation coefficients . Bilateral filtering smooths images while preserving edges by means of a nonlinear combination of nearby image values.The Laplacian filter kernel  is isotropic thus no directional information is obtained.Adding the Laplacian filtered image ′  to the image   results in a Laplacian sharpened image

Table 1 .
Percentage of crack detection completeness regarding to LF, HPF and LBP

Table 2
summarizes the mean completeness value of crack detection with respect to each variant.

Table 2 .
Mean completeness crack detection process in all 20 sample imagesThe averages of LF and LBP completeness values are almost identical.Nevertheless the completeness result of LBP would be lower if in post processing an additional closing would not be applied.Figure8visualises the percentage of crack detection completeness for the LF, HPF and LBP graphically.