HIGHWAY AND AIRPORT RUNWAY PAVEMENT INSPECTION USING MOBILE LiDAR

With an ever-increasing network of thousands of miles of pavement laid out over highways, road networks, and airport runways, their continuous monitoring is a task of utmost importance to public agencies responsible for their maintenance. The existing approaches mostly rely on a manual detection of pavement distress based on acquired image or video data – an approach that is time-consuming, costly, and whose results are subjective to the designated rater. This necessitates the need for a system that is capable of a quick data acquisition along with an efficient algorithm for the detection and quantification of pavement distress based on the acquired data. This paper proposes a LiDAR-based pavement distress detection and quantification using a mobile mapping system (MMS). Starting with a comparison of a medium-grade and high-grade MMS in terms of their accuracy and captured level of detail, this paper proves the ability of the high-grade MMS to allow the detection of shallow potholes and cracks in the pavement. Next, a fully automated algorithm is proposed to detect pavement distress from 3D point cloud followed by a quantification of the severity (in terms of the depth and volume) of the detected potholes/cracks. Finally, an experimental verification conducted over a 10 mile highway segment and two airport runway strips indicates the efficient performance of the proposed data acquisition system as well as the algorithm to report the pavement distress ranging from shallow cracks over airport runways to deeper potholes along highway segments.


INTRODUCTION
Highways, road networks, and airport runways cumulatively comprise thousands of miles of pavement laid out using asphalt, concrete, or composite materials. Routine inspection and maintenance of pavement surface is of utmost importance, especially in highways with vehicles driving at high speeds and runway strips designed for take-off and landing of aircrafts at significantly higher speeds. Detection of pavement distress is crucial for public agencies tasked with the maintenance of roadways and airport runway strips. Currently, the acquired image and video data over pavements are manually inspected by technicians on computer monitors to detect any defects. Besides this being a time-consuming and costly task, the final results are influenced by the subjectivity and the experience of the raters (Bianchini, 2010). The development of an automated procedure for pavement quality assessment and characterization is vital to facilitate large-scale pavement monitoring. A proposal for automated pavement monitoring entails two major components -(a) sensor modality for pavement data acquisition and (b) algorithm for pavement distress detection and quantification based on the acquired data. Several alternatives have been researched over the years for pavement surface condition assessment along transportation corridors (or, highways) and airport runways. A thorough review of some of the existing strategies for pothole detection along highways was conducted by Coenen and Golroo (2017) and Kim and Ryu (2014). There are four major choices for sensor modalitiesvibration-based, vision-based, thermography-based, and LiDARbasedto acquire data for pavement inspection. Each of these methods of data acquisition have been coupled with different algorithms for varying aspects of pavement surface assessment. In the area of vibration-based methods, Yu and Yu (2006) * Corresponding author developed a specialized data acquisition hardware mounted on a vehicle for conducting preliminary evaluation of pavement conditions. Their approach relied on the vertical vibration of the vehicle caused by cracks along pavement surfaces. Eriksson et al. (2008) developed a Pothole Patrol (P2) method for detecting and reporting surface conditions of roads using a vibration-based strategy. They collected data from vibration and GPS sensors to assess road surface conditions and developed a machinelearning-based approach to distinguish potholes from other road anomalies. However, in all such vibration-based approaches for pothole detection, the accuracy of detection would be lower than that from cameras or laser scanners because it can only detect potholes when the vehicle's wheels encounter a pothole. In the field of vision-based methods for pothole detection, Buza et al. (2013) proposed an unsupervised vision-based method for pothole detection, without any requirement for additional filtering or training. They adopted a three-step process consisting of image segmentation, shape extraction using spectral clustering, and identification, extraction, and roughness estimation of potholes. Zhang et al. (2014) proposed an algorithm for pothole detection using stereo-vision. They used a disparity map generated from an efficient disparity calculation algorithm to detect potholes by analyzing their distance from the fitted quadratic road surface. However, they report some false pothole detections caused by disparity calculation errors. One should note that such vision-based methods for pavement assessment are limited by the occlusion in the dataset occurring due to neighboring vehicles and certain potholes may remain undetected in these approaches. This calls for a pothole detection using a more complete dataset, which can be provided using LiDAR units onboard mobile mapping systems.
In the field of pavement assessment, especially for airport runways, thermography is a technique that has been explored over the past years. Shahin et al. (1980) developed a Pavement Condition Index (PCI) for rating jointed concrete airfield pavements. The PCI was developed as a measure of pavement structural integrity and surface operational condition based on measured distress types, severities, and densities obtained during a manual pavement inspection. The PCI was found to be correlated closely with maintenance and rehabilitation needs of airfield pavements. The applicability of infrared thermography was explored by Moropoulou et al. (2001) in order to detect delaminations in airport pavements. Their approach relied on the ability of thermography to sense the infrared radiation emitted by any structure, which subsequently allows it to detect temperature differences on the surface. They used a temperature difference threshold of 0.5˚C (determined according to standard specifications) to detect distressed areas. Another work utilizing the method of thermography was done by Tsubokawa et al. (2007), wherein they detected de-bonding between layers of flexible pavement in airports. Their experimental validation indicated the ability of infrared thermography to detect debonding at a depth of 40-70 mm from the surface of pavements depending on the weather conditions. While thermography is a technique that can detect the onset of pavement distress by detecting delamination and de-bonding in pavements, a major shortcoming of this technique is its inability to quantify the detected distress, such as the depth and thickness of delamination or detected cracks. The use of LiDAR in the field of pavement inspection is a rapidly expanding area with many agencies even providing recommendations, guidelines, or standards for LiDAR data acquisition and processing. A report published by the National Cooperative Highway Research Program in 2013 provided guidelines for the use of mobile LiDAR in transportation applications (2013). Kang and Choi (2017) used two 2D LiDAR units for extracting distance and angle information of road surface. Their approach for pothole detection using 2D LiDAR includes noise removal, clustering, line segment extraction, and finally, obtaining the gradient of the pothole. They also employed an image-based pothole detection to improve the accuracy when combined with the results from 2D LiDAR-based detection. It includes noise filtering, brightness control, binarization, edge extraction, and finally, pothole detection. The moving speed of the 2D LiDAR units used in their study was 2.7 km/hr. Moreover, the experimental verification includes only the results obtained for a single pothole detected from different positions of the mapping system with respect to the pothole. The approach by Kang and Choi (2017) deals with LiDAR units moving at extremely slow speeds and relies on individual scan line processing for pothole detection. This implies that their approach cannot be applied to mobile mapping systems moving at a considerably high speeds or LiDAR data captured by 3D LiDAR units. Laurent et al. (2008) proposed the use of Laser Crack Measurement System (LCMS) for crack monitoring using two 2D laser units. The system generates transversal profiles throughout the road segment to detect cracking and is reported to have an accuracy of 90% for transverse crack detection when the 3D crack is captured. The scope of this study was to determine if we could effectively map pothole volume and surface area at highway speeds so that patching tables can be derived. This research is focused on proposing a fully automated strategy that is suitable to conduct a two-fold assessment of pavements over highways as well as airport runway strips. The first component consists of detecting and reporting locations that exhibit any form of pavement distress, such as cracking, rutting, and potholes (occurring due to wear-and-tear, pavement patching discontinuity, missing RPMs). Secondly, the detected distress locations are characterized based on their severity, which constitutes the following attributesdepth below road surface, surface area denoting the level of degradation at different depths below road surface, and filling volume of the detected potholes. In order to achieve this end goal, we first start by introducing and comparing two different grades (medium-grade and high-grade) of LiDAR-based mobile mapping systems (MMS) in terms of the relative and absolute accuracy of their acquired point clouds. The qualitative and quantitative accuracy analysis of point clouds from the two MMS validates our choice of using the high-grade MMS for pavement distress monitoring. Finally, the proposed strategy for cracking and pothole detection and characterization is validated by testing it on a 10-mile-long highway segment and two airport runway strips. The results indicate the ability of LiDAR-based mobile mapping systems to accurately detect locations exhibiting pavement distress with varying severity and different causes.

MOBILE MAPPING SYSTEMS: DESCRIPTION AND COMPARISON
This study relies on data acquired by two mobile LiDAR mapping systemsmedium-grade and high-grade, as shown in Figure 1 (a) and (b) Ravi et al. (2018a) and Ravi et al. (2018b). The resultant accuracies of point clouds acquired by these calibrated systems are translated to their ability to capture varying level of details and detect pavement distress with varying severity. Owing to the onboard sensor specifications and system calibration, this research demonstrates that the high-grade MMS can accurately capture point clouds with a relative accuracy of less than 1 cm and an absolute accuracy of 1 to 2.5 cm. Such an accuracy facilitates the use of MMS to capture extremely minute details such as pavement tining (grooves with a depth of 3-5 mm in the pavement), which in turn, translates into its ability to detect cracking or potholes with depth as low as 1-2 cm. On the other hand, the relative and absolute accuracies for point clouds acquired by the LiDAR units onboard the medium-grade MMS is approximately 2-3 cm and 5 cm in the horizontal and vertical directions, respectively. A qualitative comparison between the two grades of MMS is conducted by illustrating two areas showing potholes with varying severity as captured from the medium-grade and high-grade mobile mapping systems as shown in Figure 2. Figure 2 (B) and (C) show a deep pothole with depth ranging upto 10 cm as captured by a high-grade and mediumgrade mobile mapping system, respectively. It can be seen that this pothole is very accurately identifiable from the high-grade system as compared to its comparatively less accurate visibility in the medium-grade system. However, a shallower pothole with a depth of 5 cm shown in Figure 2 (D) and (E) indicates that it fails to be detected in the point cloud from medium-grade system but is easily detected from the high-grade system. For the region shown in Figure 2, the road surface captured by the mediumgrade mobile mapping system has a vertical accuracy of up to 5 cm, whereas the high-grade mobile mapping system has an accuracy as high as 5 mm in data captured from a single track, which results in a better detection of potholes with varying depth/severity. While Figure 2 depicts a qualitative assessment over a highway pavement, another illustrative example over airport runway pavement is shown in Figure 3 to demonstrate the performance of the two MMS in terms of the level of detail captured by the different onboard sensors. Figure 3 shows a coarsely tined pavement with 3 to 5 mm deep grooves over an airport runway strip. The figure clearly indicates that the pavement texture is accurately captured by the Riegl and Z+F sensors, while such a detail is lost within the noise range of 2-3 cm from Velodyne LiDAR unit. This would render the mediumgrade MMS incapable of detecting shallower cracks or potholes which are less than 5 cm deep and accurately characterizing the severity of deeper potholes due to its 5 cm vertical accuracy. Based on the qualitative and quantitative comparison of the relative and absolute accuracies from the two grades of MMS along with the level of detail captured by each MMS over pavements, it can be concluded that the high-grade MMS is more competent in terms of acquiring data that can be used for the most efficient and accurate detection of pavement distress ranging from minor cracking in airport runway pavements to deeper potholes over transportation corridors (or, highways). Abiding by the above conclusion, this paper only focuses on the data acquired by the high-grade system for further assessment of the proposed pothole detection algorithm. The better absolute accuracy of the resultant point cloud from the high-grade MMS translates towards determining the accuracy in the reported location of distress, whereas the relative accuracy of the point clouds captured from different sensors is reflected in the derived quantitative characteristics, such as the surface area and volumetric estimation of the detected potholes.

METHODOLOGY
Having ascertained the accuracy of the high-grade MMS, we proceed to develop a fully automated strategy for pavement distress detection and characterization, which constitutes the following steps: 1. Road surface extraction: The road surface points are automatically extracted from the complete 3D point cloud with the aid of vehicle trajectory elevation information. The prior knowledge of the height of the onboard IMU above the road surface is used along with an additional height buffer to extract road surface points from the complete 3D point cloud. This is illustrated in Figure 4 (a), which shows the whole point cloud colored by height and colored by intensity followed by the extracted road surface points based on the proposed strategy. 2. Classification and clustering of 3D points into road surface and below-road (or, crack/pothole) points: A fully automated tile-based strategy is developed to identify all the points that lie below the iteratively obtained best-fitting plane denoting the road surface within each tile. An additional round of detection is conducted with a shifted tiling of the road surface in order to account for false detections along the edge of the tiles. The intersection of the candidate below-road points from the two tiling sequences are finally reported as the desired pavement distress points. The detected pavement distress points are labeled according to their depth below the road surface and also clustered using a distance-based region growing approach to delineate individual distress locations along the road or runway segment. These steps are illustrated in Figure 4 (b) starting from the road surface colored by intensity and colored by height followed by the two sequences of tiled road surface, which is finally succeeded by the detected distress points colored by their depth below the road surface. 3. Boundary tracing and quantitative analysis of individual distress location: Each cluster (or, distress location), as shown in Figure 4 (c), can be visualized by backprojecting the detected boundary on the corresponding RGB imagery. This would aid in identifying the source of pavement distress (such as longitudinal wear-and-tear, pavement patching discontinuity, or missing raised pavement markers). Next, each cluster is characterized by assessing the maximum depth of below-road points within the cluster, which in turn would indicate the severity of the detected distress location. Moreover, the points at different depth levels are used to trace a boundary through a minimum convex hull approach in order to derive the surface area at different depths below the road surface. The change in surface area with the depth would indicate the amount of degradation within each distress location. Finally, a volumetric estimate is computed in order to determine the type of patchwork and the amount of fill material required for fixing each distress location.
(a) (b) (c) Figure 4: Automated pavement distress detection: (a) Road surface extraction, (b) Detection of below-road points (or, pavement distress points), (c) Clustering and boundary tracing for detected distress points followed by its backprojection on RGB image The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2020, 2020 XXIV ISPRS Congress (2020 edition)

EXPERIMENTAL RESULTS
The above-mentioned approach is tested for 10-mile-long highway segment and two airport runway strips. It is observed that the proposed strategy can accurately detect potholes ranging from a depth of 2 cm (due to pavement patching discontinuity) to over 10 cm (due to wear-and-tear of pavement). Three samples of detected potholes over a highway road segment are shown in Figure 5, where Figure 5 (a) shows a pothole that is about 10 cm deep occurring due to a longitudinal wear-and-tear of the pavement, Figure 5 (b) depicts a 4 cm deep pothole as a result of pavement patching discontinuity, and Figure 5 (c) illustrates a 2 cm deep depression caused by a missing Raised Pavement Marker (RPM). The varying depth and cause of detected potholes proves the feasibility of the proposed approach for highway road network asset management. The same approach when applied to airport runway strips is seen to be able to detect cracking along and across the pavement occurring due to patching discontinuities or regular wear-andtear. Figure 6 demonstrates a sample of the point cloud captured over runway colored in grayscale based on intensity with the detected cracking (colored from blue to red based on depth below road surface) overlaid on the intensity data. These experimental results prove the ability to use mobile LiDAR systems for large-scale pavement inspection to report the location, severity, cause, and a quantified estimate of the spatial and volumetric characteristics of pavement distress. Figure 6. Cracking detected on airport runway pavement overlaid on point cloud colored by intensity

CONCLUSIONS
This paper proposed a fully automated pavement distress detection and quantification strategy applicable to highways as well as airport runway pavements. The accuracy of the derived point clouds was observed to be ± 1-2 cm using the high-grade MMS. A comparison between the medium-grade and high-grade MMS indicated the ability of the latter to conduct a fine detection of various types of pavement distresses over highways and airport runways. The proposed algorithm was seen to efficiently detect distresses attributed to varying causes, such as regular wear-and-tear, pavement patching discontinuities, missing RPMs, and shallow cracking of pavements. The applicability of a unique algorithm to pavements in different scenarios captured by an MMS at regular driving speeds indicates the versatility of the proposed research to conduct an automated pavement inspection without manual intervention.
In further stages of the study, we aim to use RGB imagery as auxiliary data for increasing the accuracy of pothole detection. Moreover, we aim to classify the potholes based on the severity as well as the root cause of the potholes, i.e., regular wear-andtear, rutting, or missing RPMs. Our future work will also be focused on denoising and improving the relative accuracy of 3D point clouds obtained from medium-grade mobile mapping system in order to facilitate an accurate pothole detection using the proposed algorithm for varying types of potholes and generate results as accurate as those obtained from a high-grade mobile mapping system.