SCANLINE NORMALIZATION FOR MMS DATA MEASURED UNDER DIFFERENT CONDITIONS

Mobile Mapping System (MMS) equipped with a high-density LiDAR scanner is widely used for mapping. Various automatic mapping methods have been proposed for point clouds measured by the high-density LiDAR scanner on the MMS. However, careful parameter tuning is often required according to measurement conditions. In this paper, we propose a method to generate normalized scanlines from point clouds captured using the MMS. Normalized scanlines are useful to avoid parameter tuning depending on the measurement conditions. In order to evaluate the validity of our method, we extracted road boundaries with the same parameters from two point clouds measured under different conditions. In our evaluation, our method could detect almost the same road boundaries from the two point clouds using the same parameter settings.


INTRODUCTION
Mobile Mapping System (MMS) equipped with a high-density LiDAR scanner is widely used for mapping. MMS allows us to make maps more accurately than conventional aerial photogrammetry and more efficiently than on-site survey.
The demand for mapping road edges has been very high due to the development of road ledger maps or autonomous driving maps, which are an important factor of the autonomous driving technology. Although these maps are drawn by human operators conventionally, there has been a problem of low productivity and variation in accuracy depending on operators' skills. Considering the growing need for high precision mapping, automation of the mapping of the road boundary is inevitable. Serna, et al. rasterized point clouds as images, in which the maximum and minimum heights were maintained as pixel values (Serna, 2013). Then, they detected road curves using the smallest elevation images. Kumar et al. rasterized the heights, reflectance values, and pulse ranges of a point cloud as an image, and extracted road boundaries from the image by using the snake model (Kumar, 2013).
However, in these methods, pixel sizes have to be carefully determined, because the quality of the image largely depends on the pixel size. While large pixel sizes lose detail shapes, small sizes lead to highly fragmented images and large computational cost.
To solve these problems, scanline-based approaches can be applied. In scanline-based approaches, Miyazaki et al. segmented a point cloud into scanlines using the angles of sequential points. Then, they detected road curves by applying region growing to line segments (Miyazaki, 2014). Cabo, et al. also generated scanlines from point clouds using time stamps. The scanlines  Corresponding author were simplified into straight lines using the method proposed in Douglas and Peucker (Douglas, 1973), and segmented the straight lines into neighbour groups (Cabo, 2016). Line groups were filtered to obtain groups of road lines, and road edges were detected from the road line groups. They reported the success rate was over 99%.
However, methods based on scanlines largely depend on intervals of scanlines. The resolutions of point clouds obtained by the MMS are typically very high on scanlines, but low between scanlines, as shown in Figure 1. The resolutions vary depending on several factors, such as scanner specifications, vehicle speed, and distances. The intervals of scanlines become large as the rotational frequency becomes large, the pulse repetition frequency becomes small, or the vehicle speed becomes large. In addition, the point density becomes low as the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B2-2020, 2020XXIV ISPRS Congress (2020 This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-325-2020 | © Authors 2020. CC BY 4.0 License. distance from the laser scanner is large. The point density also varies by the angles between laser beams and surfaces.
Since the interval of scanlines varies due to many factors, it is difficult to specify particular thresholds for scanline-based methods. Therefore, in previous methods, it was necessary to fine-tune parameters according to measurement conditions in order to obtain stable mapping results.
In this study, we propose a method to generate normalized scanlines from point clouds obtained under various measurement conditions. The proposed method enables us to generate scanlines with arbitrary intervals.
In this paper, Section 2 describes the outline and applications of the proposed method. In Section 3, we describe our method for extracting road curbs. Section 4 describes the evaluation methodology, and Section 5 shows the evaluation results. Section 6 concludes our paper and discuss future work.

Methodology
Normalized scanlines are generated from point clouds with arbitrary intervals. Our method consists of the following steps.   Normalized scanlines can be generated at any interval by changing the intervals of section planes.

Mesh generation:
Point clouds are converted into triangular meshes by connecting neighbour points. Each point cloud can be mapped onto an 2D image using the method proposed by Kohira, et al (Kohira, 2017). In this method, each point is mapped onto a pixel of a 2D image using the GPS time, the rotational frequency, and the pulse repetition frequency. In order to generate a mesh model, triangles are generated by connecting adjacent points on the image if the distances are smaller than a threshold. This method is suitable for point clouds captured using the MMS. Conventional neighbor search methods, such as kd-tree (Jon, 1975) and octree, often fail to connect points on different scanlines because of sparse point density in the region between scanlines.

Section planes:
In MMS data, the trajectory of the MMS is maintained as a sequence of coordinates. The trajectory can be represented as a polyline by connecting coordinates in order. Then, points are sampled from the trajectory polyline at equal intervals. The intervals can be arbitrary specified. Next, a section plane is generated at each point so that the normal vectors are perpendicular to the trajectory polyline. The interval between section planes becomes the interval of normalized scanlines.

Normalized scanlines:
The intersection between a section plane and each triangle becomes a line segment. Line segments can be obtained by calculating intersections between all triangles in the triangular mesh and each section plane. Normalized scanlines are generated as polylines by connecting line segments on each section plane.

Properties of normalized scanlines
Normalized scanlines have useful properties for applications such as object detection and segmentation. We discuss properties of normalized scanlines.

Perpendicular property:
Normalized scanlines are perpendicular to the MMS trajectory, regardless of the installation angle of laser scanners. Figure 3 shows the top views of the original scanlines and the normalized scanlines. The red line is the MMS trajectory. The perpendicular property is useful for extracting features arranged along the roads.  The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) intervals of normalized scanlines are constant. Therefore, it is possible to avoid parameter adjustments for feature extraction.

Up-sampling of points:
By setting small intervals for normalized scanlines, points can be up-sampled in regions with low point density. This property makes it easier to develop feature extraction algorithms.

ROAD BOUNDARY DETECTION
The properties of normalized scanlines are useful for extracting roadside features such as curbs, gutters, guardrails, planting, fences, electric wires, road markings, and buildings.
In this paper, we discuss methods for identifying the road boundaries by detecting road curbs from point clouds measured under different conditions. Figure 4 shows a process for detecting the road boundary. In Figure 4(a), the point closest to the the MMS trajectory is selected from each normalized scanline. The selected point is referred to as the seed point. Then, height changes are detected along each normalized scanline, as shown in Figure 4(b). The height change points are regarded as curb candidates. The threshold for height change is specified based on the height of curbs. Since curbs are lower than other roadside features such as guardrails, curb candidate points are searched only from points near road surfaces. The road surface height can be obtained using the MMS trajectory and the installation height of the laser scanner. The thresholds used in this paper are summarized in Table 1.
In Figure 4(c), points near curb candidates are extracted from each normalized scanline. The range for neighbour points is shown in Table 1. Then, planes are extracted from the curb candidate regions using the RANSAC method (Martin, 1981), as shown in Figure 4(d). If three planes are detected, they are regarded as the two horizontal parts and one vertical part of a curb. The thresholds for plane detection are shown in Table 1. Criteria for horizontal and vertical planes are specified as the angles from the Z axis.
Section planes for normalized scanlines are selected in the range of curb regions, as shown in Figure 4(e). The section planes are perpendicular to the MMS trajectory. Then, as shown in Figure  4(f), points on the road boundary are calculated as the intersection points among the detected planes and the section planes. The bottom and top points of the curb are detected in this step. Points on the road boundary are connected in order so that continuous curves can be obtained, as shown in Figure 4(g). The threshold for connection was specified as 2.5m. Finally, the road boundary is obtained, as shown in Figure 4(h).
Step Parameter name Value

MMS Data
We evaluated our method using actual MMS data. The MMS was the MX-8, which was developed by Trimble (Trimble websites, 2020). This MMS has two VQ-250 LiDAR scanners on the right and left of the vehicle. The MMS with MX-8 is shown in Figure  5. The specifications of the LIDAR scanner (RIEGL websites, 2020) are listed in Table 2.
Although this MMS is equipped with two LiDAR scanners, only the left side LiDAR scanner was used for evaluation. The reason The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) is to avoid measurement errors caused by the difference in measurement timing between the left and right LiDAR scanners.
Point clouds were captured in Inagi City, Tokyo, Japan. In this measurement, point clouds were captured twice with different vehicle speeds on the same route. Therefore, the two point clouds contain the same curbs, but their measurement conditions are different. We use these point clouds for evaluating our method.

Evaluation method
We evaluated our method in qualitative and quantitative ways. In the qualitative evaluation, we displayed extracted curbs on the screen, and investigated locations of successful and failed curb detection. Then, we evaluated the deviation between extracted and actual curb corners on the section view. In the quantitative evaluation, we divided point clouds along the MMS trajectory at intervals of 2m, and evaluated extracted curbs in each divided range. This evaluation was applied to the two point-clouds captured with different measurement conditions.
The following criteria were used to determine success or failure in curb extraction. First, we excluded devaluation parts from evaluation, whether such parts were extracted as curbs or not. Secondly, undetected curbs were evaluated as failures even when points on curbs were missing due to parked or running vehicles. Thirdly, when the curb location was different from actual one, it is evaluated as erroneous extraction. Figure 6 shows a curb extracted using normalized scanlines.

Qualitative evaluation
Black points indicate vertices on normalized scanlines. In Figure  6(a), two corner points are calculated as intersections among a section plane and extracted planes. In Figure 6(b), corner lines are also extracted by connecting corner points.
Figure 7(a) shows point clouds captured twice on different measurement conditions. We extracted curbs from the two pointclouds using the same extraction thresholds. We denote these two point-clouds as A and B. We generated normalized scanlines and detected curbs from each point-cloud. The extraction results are shown in Figure 7(b) and 7(c). Almost the same results could be obtained in both point clouds without threshold adjustment. Our method could stably detect straight and curved curbs, as shown in Figure 8.
We have found that extraction fails in some cases. The causes of failure cases can be categorized into five categories. The first is occlusion by vehicles. The second is the occurrence of unmeasured area due to right and left turns. The third is small steps of curbs. The fourth is the false extraction of steps other than curbs. The fifth is duplicated measurement from different directions. We describe the details of each case as follows.   (1) Occlusion by vehicle: When curbs are occluded by parked or running vehicles, points on curves are missing. The boundaries of the ground and vehicles are often incorrectly extracted as curbs, as shown in Figure 9 and 10. Figure 9(a) shows point clouds including an on-street parking vehicle. Figure 9(b) shows the road boundary extracted from point clouds. The curb occluded by the parked vehicle could not be detected. Figure 10(a) shows point clouds including a vehicle running in parallel with the MMS. Figure 10(b) shows the extracted road boundary. Occluded curbs could not be detected, and the incorrect road boundary was detected from points of the vehicle.
(2) Unmeasurable regions at crossroads: When the MMS turns left or right at the crossroad, roadsides cannot be measured. An example is shown in Figure 11. The red line shows the MMS trajectory. There are unmeasurable regions in the yellow dashed line.
(3) Small steps: The heights of curbs become small at the crossroads and the entrances of buildings. In this evaluation, the height threshold was set to 3 cm. Then, curbs with small steps partially failed to be detected. Figure 12 shows successful and failure cases for curbs with small steps. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B2-2020, 2020XXIV ISPRS Congress (2020 (4) Steps other than curbs: The borders of private lands or planting areas may be extracted as steps.
(5) Duplicated measurement: When points of the same object are measured from different sides, measured points may be misplaced, as shown in Figure 13. Figure 13(a) shows point clouds and Figure 13(b) shows a section view. In this case, points measured from different sides were not correctly aligned, and the differences were detected as steps. Table 3 shows the results of quantitative evaluation. Precision, recall, and F measure are shown in this table.

Quantitative evaluation
The accuracy of extracted boundary from point cloud A and B were almost the same. Most of the differences were due to onstreet parking vehicles and parallel running vehicles. In other words, no difference was observed depending on the vehicle speed of measurement and the distance to the target. This result shows that normalized scanlines are effective for extracting features from point clouds captured under different conditions.
Overall F measure is 93.94. The extraction accuracy at left side is more accurate than the accuracy at the right side. This is because there were many parallel running vehicles on the right side. According to the Japanese road traffic rules, vehicles are driving on the left. Therefore, it is considered that the extraction rate of curbs on the right side was low.

CONCLUSION
In this paper, we proposed normalized scanlines for enabling feature extraction without parameter tunings. In our method, the MMS point clouds were converted into mesh models, and normalized scanlines were generated as intersections between section planes and the mesh models. Section planes were planes perpendicular to the MMS trajectory. The intervals of normalized scanline can be arbitrary determined, regardless of the running speed of the MMS and the distance from the MMS trajectory. The normalized scanlines are suitable for extracting features parallel to the road like curbs, guardrails, etc.
We also proposed a method to extract the road boundaries using normalized scanlines. In our evaluation, the success rate was 94%. We also investigated failure cases for curb detection. The main causes were not point processing but the failure of measurement.
In future work, we would like to improve feature extraction.
Since we investigated the causes of failure cases in detail, exceptional procedures could be developed for each case. In addition, some steps were incorrectly detected as curbs. Deep learning approaches could be used to classify various tpes of steps  The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-325-2020 | © Authors 2020. CC BY 4.0 License.