ROBUST DETECTION OF SURFACE ANOMALY USING LIDAR POINT CLOUD WITH INTENSITY

We have developed an automatic detection method for metallic corrosion in facilities by using a LiDAR point cloud. While visual inspections for monitoring facilities are widely conducted, the inspection result depends on human skill, and there is currently a shortage of inspectors. While automatic detection methods using an RGB image have been developed, such methods cannot be applied to inspections at night. Therefore, we propose a robust detection method that utilizes both 3D shapes and intensities in a LiDAR point cloud instead of RGB information. The proposed method segments the point cloud into a basic building material by using the 3D shape and then recognizes a point cloud with an abnormal intensity in each material as the corrosion area. We demonstrate through experiments that the proposed method can robustly detect corrosion spots in aging facilities during detection conducted both during the day and at night. * Corresponding author


INTRODUCTION
While visual inspections for monitoring facilities are widely conducted, the inspection result depends on human skill, and there is currently a shortage of inspectors. In this paper, we propose an automatic inspection method that objectively identifies the anomaly area from 3D shapes and intensities in a LiDAR point cloud.
Recently, visual inspections to monitor facilities are widely conducted to prevent breakdown resulting from aging equipment (Koch, 2015). Metallic corrosion is one of the visual indicators for the deterioration level of facilities such as electric power and piping facilities. One can judge the necessity of replacing or repairing equipment by diagnosing its deterioration level on the basis of where the corrosion spots are located and the severity level for each spot. The precision of a visual inspection, which is susceptible to misdetection and/or misjudgment, strongly relies on the skill level of the inspectors. Furthermore, there is currently a shortage of inspectors, and training personnel requires a lot of time and cost (Bonnin-Pascual, 2014). In light of this situation, there is a need for a diagnosis system that can automatically and objectively specify the corrosion spots and determine whether maintenance is required.
Up to now, automatic corrosion detection has generally been performed using an RGB image taken with a digital camera. However, detection methods based on RGB information cannot be applied to inspection at night, when no workers are in the facility. Corrosion refers to deterioration on a metallic surface and is visually different from the surrounding surface. In this paper, we broadly refer to such corrosion as a surface anomaly. In the attempt to detect such anomalies, research has been done on deterioration detection for huge infrastructures, such as cracks and deteriorations in concrete, by using a point cloud with a light detection and ranging (LiDAR). A LiDAR acquires the geometrical 3D shape for a target by emitting a laser and obtains the reflected light intensity from the target surface. This process indicates where the intensity values in the deterioration spot are changed and shows the detection result based on the intensity. However, methods based on LiDAR detect the anomaly areas after manually trimming the detection target with the same building material, and therefore they cannot be directly applied to facilities that include complex structures such as pipes and steel bars.
The unresolved issues for detecting surface anomaly using the above methods can be summarized as follows.


A method based on RGB values cannot be applied for operation at night.  A method using only intensity cannot be applied for a facility with complex structures.
To tackle these issues, we propose a robust detection method that utilizes both 3D shapes and intensities in a LiDAR point cloud. By selecting intensities instead of RGBs, the proposed method can be operated even at night. Moreover, the method segments the point cloud into clusters with the basic building materials (e.g., steel bars and pipes) according to 3D shapes. In each cluster, points with an abnormal intensity are recognized as the surface anomaly. Thus, the method can be applied for a target with complex structures.
The rest of this paper is organized as follows. Related works are briefly discussed in section 2. We explain the proposed methodology of the anomaly detection in section 3. In section 4, we describe experiments in which the algorithm is applied to a terrestrial LiDAR point cloud for aging facilities and report the results. We conclude in section 5 with a brief summary and mention of future work.

RELATED WORKS
Examples of automatic corrosion detection using an RGB image include sophisticated methods that apply texture analysis, such as (Acosta, 2014) and (Medeiros, 2010), or deep learning, such as (Cha, 2018) and (Tao, 2018). However, these approaches based on RGB information cannot be applied to inspection at night because an ambient light source such as the sun or an illumination lamp is required to obtain the RGB information. Moreover, since the condition of the light changes the RGB values, other correction methodologies or training samples are required in order to ensure robust detection.
There are several methods for corrosion detection that use a LiDAR point cloud. (Aijazi, 2016) proposed a method that estimates corroded regions on the hull of marine vessels by using RGB values on a point cloud colored with LiDAR and an integrated 2D camera. After the RGB values are converted into HSV color space, points belonging to the corroded region are specified by using a histogram of the HSV values. 3D positional information is then utilized to extract the region of interest and calculate the area of the corroded regions. However, this method assumes the acquisition of RGB information, so measures must be taken to address changes of the ambient light condition, the same as with processing 2D images.
Interpreting corrosion as a surface anomaly in a broad sense, (Olsen, 2009) presented a detection method for volumetric changes and cracks in spalled concrete. The volumetric changes are estimated from positional information on a point cloud.
While this study showed that the intensity values on the cracks decreased, no automatic methodologies for mapping the crack location were described. (Maiero, 2015) proposed a semi-automated method to detect deterioration on infrastructure surfaces by means of image segmentation. The boundary of the surface deterioration is segmented by an edge detection in a 2D image with a manually defined color space referred to as lightness. After segmenting the 2D image, boundaries are mapped on the 3D point cloud. However, this method is based on RGB information, so the concerns regarding changes to the ambient light remain. This paper also discussed the availability of the intensity, which is a reflection amplitude of the laser at the measured point. If we assume that this method including the usage of intensity is applied for monitoring a facility with several structures in one scene, every boundary between the structures is recognized as a boundary of the deterioration surface.
When a surface anomaly causes bumps or distortions, spatial roughness is a candidate to evaluate it numerically. (Turner, 2014) evaluated the roughness of soil surfaces on the order of the centimeter by using a point cloud with an airborne LiDAR. Corrosion generally has a thickness on the order of the submillimeter, while a high-precision LiDAR reaches only the order of the millimeter. Therefore, the roughness cannot be used to identify the deviation of surface thickness induced by the corrosion. When severe corrosion distorts a surface's shape, roughness is a useful indicator. However, in this work we focus on corrosion without shape changing.

DETECTION OF SURFACE ANOMALY
The proposed method detects areas with a surface anomaly by determining the points with an abnormal intensity in each cluster, which is segmented into the same building material according to 3D shapes. The intensity measured with a LiDAR depends on the characteristics of the absorption and the scattering of the laser for the irradiated surface. When a surface has an abnormal area including corrosion, the intensity values of the corresponding points temporally vary from a normal intensity, and it has a different value from the surrounding points. Our method detects the surface anomaly by utilizing the spatial difference of the intensity from surrounding points.
Radiometric calibration based on the radar equation is widely used as a classification method that utilizes the LiDAR intensity. The reflected light intensity Pr is given by (Briese, et.al, 2012) (1) with transmitted laser power Pt, the diameter of the receiver aperture Dr, the range between sensor and target Ri, the backscattering coefficient of the surface i, and the atmospheric and system transmission factors atm, and sys respectively. All parameters except Ri and i are constant during the scanning of a facility by terrestrial LiDAR. The surface anomaly appears in i, which strongly depends on the incident angle of the laser to the target surface. Therefore, the dependencies of Ri and the incident angle of the laser have to be considered. Figure 1 shows the processing algorithm of the proposed detection method. It consists of four blocks: "Clustering", "Grouping clusters", "Sub-clustering by incident angle", and "Decision of anomaly points". The first three parts segment a point cloud into the basic building materials with approximately the same Ri and surface material (e.g., steel bars and pipes), along with the incident angle. The final one determines the anomaly points on the basis of the intensity. The processing details are explained in the following section.

Clustering
This process segments the input point cloud into the basic building material for each object in the scene. In this paper, we use region-growing clustering (PCL, 2016) for this process. The regions are grown from the angle between the normal vectors and the curvature differences from neighbor points. In the case of monitoring facilities, the inspection targets including corrosion are man-made objects. Such objects typically have a bump at the boundary between the basic building materials created by their joints. Thus, the region growing based on the discontinuity of normal vectors is suitable for the segmentation. Another general clustering method, Euclidean clustering (PCL, 2016), which segments the point clouds located near each other into a cluster, is not suitable for this purpose because it often segments several building materials into the same cluster. Figure 2 shows the point cloud used in our explanation from here through section 3. Each cluster is filled with a different color, so the different steel bars and the pipes can be correctly segmented. Figure 2. Example of region-growing clustering for a facility with complex structures.

Grouping clusters
This process groups several clusters segmented by the clustering algorithm in the previous section, although they originally belong to the same building material. The point cloud of a facility with complex structures includes many occlusions. These occlusions split a building material into several clusters according to the region growing, so structures existing far from the sensor are subdivided into excessively small clusters. Our technique requires the cluster to be larger than the anomaly so that an area of lower intensity can be detected in the corresponding cluster. Therefore, this process estimates combinations of the clusters subdivided by the occlusions and recognizes them as a single cluster.
The selection criteria of grouping clusters assumes a man-made object as the target. The following criteria are applied to all combinations of clusters.
1. Two clusters are located on the same plane.
2. Both clusters have common edges. 3. Projected clusters exist within a short distance of each other. 4. Occluded region exists between the clusters.

Clusters on the same plane:
The first process skims the combinations of clusters. When the split clusters belong to the same material, they mostly exist on the same plane. Thus, a criterion confirms whether the two clusters are located on the same plane after each cluster is approximated by a plane. The approximated plane of a cluster is defined as the plane located at a centroid of the cluster and perpendicular to the third vector of a principal component analysis (PCA). The combination of two clusters is selected as the candidate for the grouping, when an angle between the third vectors of the PCA and an averaged distance between the plane and points in another cluster is less than a certain threshold. By approximating the cluster by a plane, even curved objects (such as pipes) can be segmented.

Existence of a common line for cluster edges:
Since a man-made object tends to be rectilinear in shape, the edges of the split clusters exist in common lines. The fitted line of the edges for each cluster is regarded as the common line, when both an angle between the edge lines and an averaged distance between the line and edge points in another cluster are less than the threshold values, respectively. In this paper, more than two common lines are required for the grouping candidates. An exception is when one common line is detected, in which case the subsequent selection scheme described in section 3.2.3 is applied.

Distance between projected clusters:
When one of the edge lines disappears due to the occlusion, the criterion shown in section 3.2.2 drops off some candidates. Therefore, when one common line between the cluster edges is detected, the following selection scheme is applied. After projecting clusters along the common line, the distance between projected clusters is evaluated. In this paper, the distance is defined as an averaged value of closest distances between projected clusters.

Existence of an occluded region between clusters:
The final process confirms whether occlusions have split the clusters for the remaining candidates. After generating a point cloud that interpolates between the clusters using the common edge lines, the candidates remain when the point cloud exists between the interpolated point cloud and the sensor origin. Figure 3 shows an example of the cluster grouping described so far. In addition to the measured point cloud, the interpolated point cloud for the remaining combination candidates is shown as green points. The green points accurately fill the region occluded by the front objects, so the grouping algorithm can group the split clusters as a single cluster successfully. Figure 3. Result of cluster grouping. White points indicate "measured" and green points indicate "interpolated".

Sub-clustering by incident angle
For each cluster, this process segments the cluster into several sub-clusters whose points have a similar incident angle. The intensity acquired by a LiDAR depends on the distance between the sensor origin and the measured point, and also on the incident angle of the laser to the target surface. In the case of monitoring facilities, the former factor rarely changes the intensity value unless the cluster is sufficiently long in the depth direction. However, the latter one is a prominent issue in monitoring facilities, and pipes are the major examples. Figure  4 shows a point cloud coloring only a cluster of the pipe with a gradient according to the intensity. Since the incident angle becomes shallow on the side of the pipe due to its cylindrical shape, the intensity on the side region decreases. Therefore, the intensity in the cluster is drastically changed by not only the surface anomaly but also the incident angle.
In order to mitigate the dependency of the incident angle for determining the surface anomaly, an additional process divides the sub-clusters with a similar incident angle. The incident angle of each point is estimated from an angle between a vector from the sensor origin to the measured point, and the normal vector. According to the estimated incident angle, the points in the cluster are divided into sub-clusters. Figure 5 shows the sub-clustering result for a pipe and a flat structural steel. While the flat steel consists of one sub-cluster, the pipe is divided into several sub-clusters.

Decision of the anomaly points
This process, which is the final step, determines the points with abnormal intensity values in the sub-cluster. The intensities for points in a sub-cluster are almost the same unless the cluster contains an anomaly. Thus, the points separating from the normal intensity value in a sub-cluster can be regarded as areas that contain a surface anomaly. In this paper, we fix the range of the normal intensity for each sub-cluster on the basis of the histogram of the intensity. Figure 6 shows an intensity histogram of a sub-cluster that includes a partially corroded area. After fitting a normal distribution to the histogram, the intensity value deviating from the distribution is recognized as the surface anomaly. Figure 6. Histogram of intensity for partially corroded cluster.

EXPERIMENTS
We performed experiments using a LiDAR point cloud in order to evaluate the effectiveness of the proposed detection method. The measurement setup and the analysis parameters are presented in section 4.1, the experimental results and comparison with the conventional method are described in section 4.2, and the verification results for the robustness are shown in section 4.3.

Experiment condition
We captured aging facilities as the experiment target using a terrestrial LiDAR. A FARO FocusS350 was placed 5-20 m from the target and obtained point clouds 0.035 degrees per point to reach a sufficient number of points for analysis.
The parameters used for the proposed method are summarized in Table 1. For region-growing clustering (discussed in section 3.1), a radius for estimating a normal vector and two threshold values for recognizing same clusters must be specified. We set the former radius to a relatively large value to prevent dividing into excessively small clusters. The "Grouping clusters" process requires the angle and the distance for identifying whether two clusters are located on the same plane, along with a threshold value of the average distance between the projected clusters. In the "Sub-clustering by incident angle" process, a cluster is uniformly divided into sub-clusters according to a cosine value of the incident angle. The bin width of the histogram in the "Decision of anomaly points" process is selected according to the intensity resolution of FARO FocusS350. The abnormal intensity is defined as three sigma away from the fitted normal distribution in each sub-cluster.

Comparison with conventional method
In this section, we compare the surface anomaly detection result by the proposed method with that of the conventional method. Figure 7(a) shows the RGB image of the experiment target captured by a digital camera. The target included un-painted corrosion areas everywhere. The unnecessary part in the background has been removed from the figure. Figure 7(b) shows the point cloud coloring according to intensity strength.
As we can see, the intensity in the corroded region relatively decreased compared to the surrounding area. The point cloud belonging to the ground was removed since the ground is not the monitoring target. As the conventional method uses only the intensity, Figure 8(a) shows the result where the anomaly points are determined by thresholding a uniform intensity without the clustering. The red points indicate the detected surface anomaly. The threshold value was adjusted to detect the corrosion on the front structural steel. Besides the corrosion spots, the red points appeared everywhere since this facility includes several kinds of building material. For example, the surface with a shallow incident angle, the grayed small-bore pipe, and the plants reflected comparably lower intensity in this scene, and they were detected as the surface anomaly. The detection result by the proposed method is shown in Figure 8(b). In addition to the visual corrosion spots in Figure 7 (a), the spots on the obscure back structure were successfully detected. However, a few false positives remained in the frame surrounding the fence. These were caused by a failure of the region-growing clustering, which segmented the frame and the fence as the same cluster.
Next, to confirm that our method was not affected by ambient light, we performed measurements for the same target again but this time at night. Figure 9(a) shows the RGB image captured by a digital camera. Since only the light at the front-left and the fluorescent lighting in the buildings were visually confirmed, corrosion spots could no longer be extracted from this image. Figure 9(b) shows the detection result of the proposed method. The LiDAR obtained the point cloud at the same time when the image in Figure 9(a) was captured. Partly because the LiDAR was located at the same position as it was in Figure 7, the same corrosion spots as in the daytime detection were successfully detected.

Robustness
This section demonstrates the robustness of the proposed method through two experiments: one where we changed the viewing angle to the same target, and the other where we changed the target itself.
In the first experiment, we investigated whether the proposed method could extract the same anomaly area from the point clouds captured from a different viewing angle. Due to the intensity dependency on the incident angle (described in section 3.3), the intensity values change when the point cloud is captured from a different observation point. Figure 10 shows the point cloud colored according to the intensity from the different observation points. We can see that the intensity difference with and without corrosion is significant. Figure 11 shows the results when we applied the proposed method to these point clouds. As we can see, it extracted almost the same anomaly area among all observation points. However, the result of observation point 1 (Figure 10(a)) missed the corrosion area on the upper-right pipe. As shown in Figure 10 (a), the intensity values in the missing area were almost equal to those of the surrounding points. Since the intensity gradient to the incident angle for the surface without corrosion was larger than the one with corrosion, the intensities in regions with shallow incident angles were close to each other. . Detection results by proposed method for scene in Figure 10.
In the second experiment, we investigated whether the proposed method could extract the surface anomaly for different targets. Figure 12 shows the RGB images of the new experimented target, which includes corrosion on pipes and structural steels. Figure 13 shows the results of applying the proposed method to this target. Both the corrosion spots visible in Figure 12 and the shadowed region in Figure 12(a) were also successfully detected.
However, false positives appeared in the back structures due to a failure of the region-growing clustering.
We should point out here that the area detected as the surface anomaly is inconsistent with corrosion color in RGB images. Especially for corroded areas on a pipe, the detected area is mostly narrower than the area with corrosion color. This difference is usually caused by rust leachate, which means the intensity-based detection can specify the source area of corrosion.

CONCLUSION
We have proposed an automatic detection method that specifies the area with a surface anomaly by using a LiDAR point cloud.
Our method extracts the anomaly area for a wide-area point cloud including several complex structures by using the intensity values in each building material. Its processing consists of region-growing clustering, a grouping algorithm for the split clusters caused by occlusions, and a sub-clustering based on the incident angle of the laser to the target surface. Experimental results showed that the proposed method robustly detected the area with a surface anomaly in an aging facility both during the day and at night. However, a few false positives appeared, mainly due to a failure when clustering building materials. A dedicated clustering algorithm for man-made objects will be required.