EXTRACTION OF ROAD EDGES FROM MLS POINT CLOUDS USING BEND ANGLE OF SCANLINES

Efficient road edge extraction from point clouds acquired by Mobile Laser Scanning (MLS) is an important task because the road edge is one of the main elements of high definition maps. In this paper, we present a scanline-based road edge extraction method using a bend angle of scanlines from MLS point clouds. Scanline-based methods have advantages in that computational cost is low, it is easy to extract accurate road edges, and they are independent of driving speed of MLS compared to methods using unorganized points. In contrast, there are some problems with these methods where the extraction accuracy becomes low at curb cuts and intersections. The extraction accuracy becomes low caused by the scanning noise and small occlusion from weeds and fallen leaves. In addition, some parameters should be adjusted according to the mounting angle of the laser scanner on the vehicle. Therefore, we present a scanlinebased road edge extraction method which can solve these problems. First, the points of the scanline are projected to a plane in order to reduce the influence of the mounting angle of the laser scanner on the vehicle. Next, the bend angle of each point is calculated by using filtered point clouds which are not vulnerable to small occlusions around the curb such as weeds. Then, points with a local maximum of bend angle and close to trajectories are extracted as seed points. Finally, road edges are generated by tracking based on bend angle of scanlines and smoothness of road edges from the seed points. In the experiments, our proposed methods achieved a completeness of over 95.3%, a correctness of over 95.0%, a quality of over 90.7%, and RMS difference less than 18.7 mm in total.


INTRODUCTION
Point clouds acquired by Mobile Laser Scanning (MLS) have been applied to efficient road asset management and the improvement in accuracy and generation cost of high definition maps. Road edges, which are defined as border lines between a roadway and sidewalks, are one of the most important information points in these applications. Therefore, many methods for automatic extraction of road edges have been proposed (e.g. Qiu et al., 2016, Hervieu et al., 2013, Zai et al., 2018, Yang et al., 2013, and Ishikawa et al., 2018. In most proposed methods, road edges are acquired based on curb extraction by evaluation of local points distribution. These methods can be classified into unorganized points-based methods and scanline-based methods by evaluation processes of local points. As the unorganized point-based methods, Qiu et al. (2016) extracts candidate points of road edges from multiple planes extracted by RANSAC algorithm, and extracted road edge points from candidate points based on the stability and continuity of the road width. Hervieu et al. (2013) proposed a method to detect curb points using Kalman filter inspired method from roadside points extracted by evaluation of the difference of normal vector and estimated plane direction using RANSAC. Hernández et al. (2009) extracted the road region and the road boundary by evaluation of height difference of points using height images. Rodriguez-Cuenca et al. (2016) generated images which recorded height difference and number of points in each pixel from point clouds and extracted curb candidate pixels by evaluation of these images. Furthermore, curb points are * Corresponding author extracted based on the evaluation of the distance between the plane estimated by the least squared fitting and each point using points corresponding to the curb candidate pixels. Zai et al. (2018) proposed a road edge extraction method using Supervoxels with several attributes such as normal vector and intensity of point clouds and applied the α-shape algorithm and the graph cuts-based energy minimization algorithm to Supervoxels. The advantage of these methods is their applicability in any point clouds. However, the disadvantages are that the computational cost is high due to iteration of points search processes at the calculation of normal vector, these methods are weak to changes in points density due to the specification and driving speed of MLS because they are based on processes and parameters depending on point density such as using images with a certain resolution, and it is difficult to extract road edges accurately when road edges are not parallel to the trajectories because these methods adopt the parallel condition from the trajectories when road edges are extracted from candidate points.
In contrast, scanline-based methods have advantages such as the low computational cost compared with the ones using unorganized point clouds, it is easy to extract accurate road edges which are independent of driving speed of MLS. As scanlinebased methods, Yang et al. (2013) proposed a semi-automated extraction method of road boundaries. In this method, curb candidate points are extracted based on moving operation of windows which consist of a certain number of neighboring points on scanlines, and curb points are extracted by optimization of segmented curb candidate points. Miyazaki et al. (2017) proposed a scanline-based region growing method for extracting flat regions such as roads, curbstones and sidewalks. In this method, point clouds on the scanline are approximated to polylines and planar regions are generated by region growing based on the evaluation of geometric distance and normal vector of each polyline. Ishikawa et al. (2018) extracted curb points and classified curb types from low-density points by angle evaluation of neighboring points on scanlines. In this method, extracted curb points were refined by using evaluation of point distribution and a Statistical Outlier Removal (SOR) filter. However, these methods have some problems; the extraction accuracy is low at curb cuts and intersections because these road edges have only a slight height difference and it is difficult to adjust the parameters. Furthermore, extracted lines may be influenced by the mounting angle of the laser scanner on the vehicle, weeds and fallen leaves around the curbs because neighbor points on the scanline are simply used for angle evaluation.
The objective of our research is to develop an automatic extraction method of the road edges passing through the lower edge of the curb from the point cloud acquired by MLS in order to solve these problems. Our method is based on tracking of road edges using bend angle, and are not vulnerable to slight occlusions around the curb by using filtered point clouds.

Outlines
The majority of our method uses scanlines. In our method, we In our method, road edges are obtained by tracking based on bend angle of each point calculated from scanlines and smoothness of road edges from seed points which are considered as curb edges. In order to improve the stability of the bend angle calculation for each point, projection of point clouds, irregular point removal, scanline smoothing, and bend angle estimation based on neighbors at a certain distance are used. This method consists of (a) projection of points to the driving direction, (b) calculation of bend angle, (c) removal of irregular points, (d) smoothing of scanline, (e) extraction of seed points of road edges, and (f) extraction of road edges shown in Fig. 1. Steps (a)-(d) are processes of each scanline. The scanline is defined as a point set which is divided at the upper of trajectories and is ordered by acquisition time. The details of each step are described below.    The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition)

Projection to the Driving Direction (Fig.1 (a))
The points of the scanline are projected onto a plane perpendicular to the driving direction of the vehicle for detection of the curb edge without the influence of the mounting angle of the laser scanner on the vehicle, as shown in Fig. 2. The coordinate of point projected on the plane perpendicular to the driving direction is obtained by Eq. (1) and (2).
where ( , ), = coordinate and yaw angle of trajectory at the acquisition time of the point

Calculation of Bend Angle (Fig.1 (b))
The degree of bending of the scanline becomes large at the edge of the curb, and becomes small at the curb cut because the cross slope is designed to change at the border between the roadway and the sidewalk. In order to recognize the degree of bending at each point, the bend angle is defined as shown in Fig. 3. For reducing the influence of measurement noise, first, two neighbors, and , of an interesting point (position: ) are set as farthest points within the fixed distance from (we assume that < < , where is the acquisition time of point ). Then, the angle from the line passing through the points and to the line passing through the points and is calculated as the bend angle of the point . If the shape of the three points used in the calculation is concave towards the position of the laser scanner, the sign of the bend angle is positive, otherwise the sign of the bend angle is negative. As shown in Fig. 4, robust computation of bend angles for reducing the influence of measurement noise is realized by using the farthest point within the fixed distance. (Fig.1 (c)) Irregular points such as plants near the road edge may cause false extraction of road edge points in the following steps because absolute values of the bend angle of these points are relatively high. Therefore, points where the absolute value of the bend angle is larger than a given threshold are removed. As shown in Fig. 5, even if curbs are covered by plants, it is easier to extract road edges with the following steps because most of the irregular points are removed at this process.

Smoothing of Scanline (Fig.1 (d))
Because tracking processes of road edges in the following steps is sensitive to slight changes of the bend angle, it is necessary to reduce the influence of measurement noise. In our method, Taubin smoothing (Taubin, 1995) is applied to the scanline, and the bend angle is calculated again using the smoothed scanline.
Taubin smoothing is a removal method of measurement noise by low-pass filtering which repeats the Gaussian smoothing step while exchanging positive and negative scale factors. The position ̂ +1 of the new points after the one smoothing step is defined as shown in Eq. (3).
where ̂ = position for the -th smoothing = weight = scale factor The weight parameter = .5 is used as the reciprocal of adjacent point number. Scale factor = .63 7 is used for the even number of smoothing steps, and = − .6732 is used for the odd number of smoothing steps based on (Taubin, 1995). We applied Taubin smoothing to the scanline around a curb to determine the optimal number of repeats. The smoothing results by different numbers of repeats to the actual scanline are shown in Fig. 6. In the original data, a noise point exists at the side of the curb on the scanline. As the number of smoothing increases, this noise point mixes with the side of curb points and becomes less noticeable while the shape of the curb is maintained. As the result of experiments, we determined that the suitable number of repeats is 20.

Extraction of Seed Points of Road Edges (Fig.1 (e))
In our method, the road edges are tracked from the position of seed points considered as a curb edge. This section describes the extraction procedure of seed points. First, points whose bend angles are local maximum within the range ( , ) are extracted as peak points. The range is determined to obtain only curb points at which the bend angle is close to 90 degrees. Next, we track the scanline from the point directly below the trajectory to the outside of the road, and the first peak point is extracted as a seed candidate point. It is considered that the seed candidate points include non-curb points in the cross section of the road corresponding to the scanline without curbs such as curb cuts and intersections. Therefore, lower-edge points of curbs are extracted from the seed candidate points by using the property that local seed candidate points can be linearly approximated if seed candidate points consist of only curb points.
Seed candidate points are first divided into left and right point sets of the trajectory. Then each point set is also divided along the trajectory using the given intervals . Line fitting using RANSAC (Fischler et al., 1981) is applied to each point set, and the closest point to the fitted line is set as the seed point. In this process, there is no limitation by the direction of the fitted line in order to extract seed points even if the curb is not parallel to the  The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) trajectory. Therefore, non-curb points such as boundary points between a vehicle and a road surface may remain in seed points, but those points are removed after the extraction of road edges. (Fig.1 (f

))
The road edge (point sequence) is obtained by iterative connection of neighboring points from seed points. As shown in Fig. 7, the point of road edge is selected as the one with minimum energy in a local point set which is located around the search distance along search direction from a current road edge point. The energy evaluates the bend angle and smoothness of the road edges, and the next road edge point is determined using Eq. (4).
where , = average and standard deviation of the bend angle in the point set ∆ ℎ ( ), ∆ ( ) = horizontal and vertical angle between the search direction and the vector connecting the point and the previous road edge points The initial search direction is the line direction calculated by RANSAC in Sec. 2.6, and the following direction is defined by weighted difference vectors of previous road edge points, as shown in Fig.8 and Eq. (8).
If there are no points in the search range, or there are only points with the bend angle less than the given threshold in the search range, the extraction of road edges is finished.
After extraction of road edges, short road edges are removed. Next, in order to remove edges between road and objects in the roadway, such as vehicles, if the points exist in a certain height range above the road edges, the edges are removed. Furthermore, to remove edges outside the roadway, if the line segment perpendicular to the extracted line , whose endpoints are an endpoint of and its nearest trajectory point, intersects with another extracted line, is removed.

Experimental Data
The point cloud used in the experiments was collected using an MLS named StreetMapper 360. The specification of MLS and the acquired point clouds information is shown in Table 1. In these experiments, only the point cloud acquired from one laser scanner is used.   The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) of points around the other side of road edges is approximately 120 mm.

Quantitative Evaluation
The extraction results of road edges are shown in Figs.9 and 10. Fig. 9(a) and Fig.10(a) are overviews of the extracted road edges by the proposed method, Fig.9(c) to (g) and Fig.10(c) to (f) are enlarged views of the extracted road edges, and Fig.9(b) and Fig.10(b) are the manually extracted road edges for evaluation. It is observed that road edges of curbs, curb cuts, and intersections were extracted except for occlusion parts of road edges. At the intersections, road edges are extracted until just before the position where points are sparse and road edges become unclear. It was observed that the road edges are extracted without the influence of weeds that exist around curbs such as shown in Fig.  9(f). The road edges were extracted, even the parts where road edges are not parallel to the MLS driving direction such as the bus bay, as shown in Fig. 9(g). On the other hand, when the direction of scanlines and road edges are parallel, the length of extracted lines at the intersection tends to be short.
The performance of the proposed method is quantitatively evaluated using Heipke's proposed method (Heipke et al., 1997) which has been widely used in related works (e.g. Zhou et al., 2012). This method evaluates extracted lines using the ground truth data as the reference lines. Therefore, we manually extracted road edges in the point cloud as the reference lines. The reference lines were not generated at the occlusion part, such as behind a vehicle.
When a buffer is generated from the reference line, True Positive (TP) is defined as the extraction line included in the buffer, and False Positive (FP) is defined as the extraction line outside the buffer as shown in Fig. 11. Similarly, when a buffer is generated from the extraction line, False Negative (FN) is defined as the reference line outside the buffer as shown in Fig. 11. Then, completeness, correctness, and quality are defined as shown Eq. This evaluation method requires a suitable setting of the buffer width because the value of the buffer width significantly affects the evaluation results. We adopted the buffer width of 50 mm because the distance from the extracted line to the true road edge was often kept within approximately 50 mm in the case of manual extraction. This buffer width is small compared to related works, for example, Zhou et al. (2012) adopted 500mm, Qui et al. (2016) adopted 200 mm, andKumar et al. (2013) adopted multiple values ranging from 100 to 500 mm as the buffer width.
In order to simplify the calculation of each quantity, sampled points in 10 mm intervals from each line are used in the calculations. The extracted road edges were divided into curbs, curb cuts, and intersections of the left and right sides of the road, and were evaluated individually.
The results of the quantitative evaluation are shown in Tables.2 and 3. The results are summarized as follows.
Completeness, correctness and quality: Our proposed methods achieved the completeness of over 97.1% and 89.9%, the correctness of over 97.2% and 89.0%, the quality of over 94.4% and 80.9% in total at the site of Kawasaki city and Sapporo city. It is considered that the results of Sapporo city were relatively low accuracy because of the curb height is lower than in Kawasaki city. There were no extracted lines of other than road edges in both sites. The extraction quality of left and right road edges were similar, 94.4% and 99.1% at Kawasaki city. In contrast, the extraction quality of right road edges were lower than left road edges at Sapporo city because the extraction quality is more influenced by point density when curb height is low as shown in Fig. 10(e) and (f). The quality of curb cuts was high similar to the quality of curbs on both sides. On the other hand,   Figure 11. Evaluation method The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B2-2020, 2020XXIV ISPRS Congress (2020 at the intersections, the quality achieved over 62.7% which is lower than the one of curbs or curb cuts because the height difference is small and road edges curved sharply. RMS difference (α): The RMS difference α is less than 18.7mm in total. These values of the left side where point density was high tend to be lower than the right side. The RMS difference α of the left side is close to the point intervals around the road edges, and one of the right side is smaller than the point intervals around the road edges. These results indicate that optimum points were selected as road edge points at the tracking process by energy function.
Number of Gaps ( ) and Total gap length (ℎ): The number of gaps were 24 and 86, and total gap length ℎ were 17.2 m and 69.9 m in total at the site of Kawasaki city and Sapporo city. These values tend to be better for curbs and curb cuts than intersections.
The processing time for approximately 18 million points at the site of Kawasaki city was 74.8 seconds, and the processing time for approximately 17 million points at the site of Sapporo city was 74.3 seconds on a PC with CPU Intel Core i7-7700K and 32 GB RAM.

CONCLUSIONS
In this paper, we proposed an accurate extraction method of the road edges from point clouds using the bend angle of the scanline acquired by MLS. Our proposed method extracts road edges without the influence of the mounting angle of the laser scanner on the vehicle, and results in makes robustness against slight occlusions around the curb by using filtered point clouds. In addition, the road edges at curb cuts and intersections where the height difference is small at edges can be extracted using tracking based on the energy defined by bend angles of scanlines and smoothness of the extracted line.
In the experiments using two point clouds, the completeness of over 95.3%, the correctness of over 95.0%, the quality of over 90.7% are achieved in total. Our proposed method can extract at similar quality although we used 50 mm buffer width which is smaller than related works. RMS difference less than 18.7 mm was achieved in total. This result indicates that our proposed method is very accurate. At the intersections, road edges were extracted until just before the position where points are sparse and road edges become unclear. Furthermore, our proposed method also indicated good performance at road edges where the point density is low. Future studies will include an automatic classification of curbs or curb cuts and accuracy improvement using point clouds from two laser scanners on the MLS.