A CALIBRATION METHOD OF TWO MOBILE LASER SCANNING SYSTEM UNITS FOR RAILWAY MEASUREMENT

This paper proposes a methodology to calibrate the laser scanner of a Mobile Laser Scanning System (MLS) with the trajectory of the other MLS, both of which are installed directly above the top of both rails. Railway vehicle laser scanners systems of MLS are able to obtain 3D scanning map of the rail environment. In order to adapt the actual site condition of the maintenance works, we propose a calibration method with non-linear Least Mean Square calculation which use point clouds around poles along rails and sleepers of rails as cylindrical and planner constraints. The accuracy of 0.006m between two laser point clouds can be achieved with this method. With the common planar and cylinder condition Leven-Marquardt method has been applied for this method. This method can execute without a good initial value for the extrinsic parameter and can shorten the processing time compared with the linear type of Least Mean Square method. * Corresponding author


INTRODUCTION
Laser scanning technology is becoming more and more common. Methods such as Terrestrial Laser Scanning System (TLS) and Mobile Laser Scanning System (MLS) have begun to be used for monitoring railway infrastructures. TLS can acquire point cloud for the railway tracks with high density although it would take much time to acquire scanning data because of the long structure of the rail. MLS began to emerge as the practical application for the railway environment thanks to its high efficiency and comparable accuracy (Soni et al., 2014, Stain, 2015, Elberink, Khoshelham 2015, Niina et al., 2018. However if the moving speed is increased, the sufficient point density would not be obtained. The authors proposed a method of deploying two MLS units directly above the rails and of matching two sets of point clouds based on the centre line between railway gauges (Yamamoto, Yabuki, 2019a, 2019b. Applying with this method, sufficient point cloud density can be obtained by matching the point clouds of the two MLSs directly above the rails based on the centre position between the gauges. However, it remains the problem that the position accuracy of the point cloud depended on the extraction situation of the centre line between gauges. This problem occurred when the MLS units were installed at front and rear of the road-rail vehicle to eliminate the occlusion of the laser point cloud in the traveling direction. Since the laser scanners mounted on the front and rear were installed at different units of the vehicle, the position accuracy of the centre of the gauge was affected by rocking and vibration when the vehicle passed the rail joints and curves. In order to apply multiple MLSs to the railway environment and to ensure the accuracy and efficiency in the acquisition of point cloud data, it is necessary to calibrate these MLSs to obtain matched point cloud. For the indoor usage the multiple laser scanners calibration were conducted (Jung et al., 2015) In the outside field it is common to use the surroundings as a common target (Rieger et al., 2010). Moreover, for more precise calibration large targets are utilized (Heinz et al., 2017). They use planar target along the course of data acquisition with various angles toward the longitudinal direction. The target are captured by the TLS to acquire better accuracy (Hong et al., 2017). The approach of plane-based method for MLS is a highly practical and reliable. Therefore, in this study we also take plane feature as a constraints for the calibration. Besides the planar target a pole with reflective tape (Underwood et al., 2007) and multiple reflective markers with an efficient secondorder cone-based optimization (Gao, Spletzer, 2010) were applied in known locations in an environment with a flat ground plane. However, at the survey site the additional procedure beforehand such as a settlement of target marker should be avoided. The entropy based self-calibration method which minimize the estimated value of relative orientation from multi directional trajectory. (Hillemann et al., 2019, Maddern et al., 2012. However, because the trajectory of MLS on the railway environment would be along the rail the environment cannot allow the data acquisition of the multi-directional trajectory. Therefore, many self-calibration approach are conducted for vehicle-based MLS taking advantage of the character that the vertical transformation and variation of roll and pitch angle are limited. In the railway environment the combination use of planar and standing pole is one of the practical way. In this paper, we propose a calibration method applying the mutual orientation method based on the collinear and coplanar condition of photogrammetry in order to stably match the point cloud of the two MLSs directly above the rails. This contribution is organized as follows. In Section 2 we present the proposed method for the calibration of two MLSs system units directly above the rails. In Section 3 to support the theoretical considerations we conduct experiments in the actual rail situation equipped with electrification pole and rails sleeper for the constraints. In Section 4 we discuss the results and Section 5 we concludes this contribution.

CALIBRATION METHODOLOGY
In order to calibrate the relative orientation between two MLSs directory above the rails we propose the method with non-linear Least Mean Square calculation which use point clouds around poles along rails and sleepers of rails as cylindrical and planner constraints.

Overview of proposed method
In this work, we propose the system calibration method which comprises the determination of the range offset and the extrinsic calibration parameters of a laser scanner of MLS (MLS-b) from an origin of another MLS (MLS-a). Figure 1 shows the workflow of the calibration and serves as an overview of the method. The inputs composed with three parts, First, inputs from MLS-a to the calibration are POS data and 3D point cloud which has georeferenced coordinates. From the 3D  (Fischler and Bolles, 1981). The segments consist of the 200 points specified as the sleepers of the rail and the poles. Second, inputs of MLS-b to the calibration are raw data of laser scanner and time information which can be calculated to GNSS time. GPS time should be adjusted Third inputs are the initial extrinsic parameters and reference parameters extracted from the planar and cylindrical segments. Two MLSs are deployed on the railway vehicle as shown in Figure 2. Equation (1) shows the transformation of a scanned point from the local coordinate frame of the 2D laser scanner to the geographical coordinate frame of the positioning and orientation solution (POS).

POS
Laser POS Where   ,, x y z =coordinates on the plane or cylindrical segment.

Creating the reference using initial extrinsic parameters
Using with initial extrinsic parameters 3D point cloud are reconstructed according to equation (1). Since MLS-a and MLS-b are independent systems the time cannot completely meet the timing. Therefore the time for the reconstruction should be interpolated for the timing for the laser pulse. Equation (2) Since the proposed method uses non-linear Least Mean Square calculation the evaluation criteria should be given with the feature measurement. The difference between 3D point cloud of MLS-a and MLS-b should be minimized during iterative calculation loop for the optimization. In the limited trajectory environment which MLS vehicle is on the rail we propose utilize sleeper of the rail as the planar constraint and pole standing along the rail as the cylindrical constraint. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2020, 2020 XXIV ISPRS Congress (2020 edition) Equation (3) shows the model of the plane which is extracted from point cloud of MLS-a and MLS-b.

F Ax By Cz
,, x y z = coordinates of the plane. Therefore, the value for evaluation for matching between point cloud of MLS-a and MLS-b are calculated with equation (4) 2 2 2 2 1 1 1 , ,, x y z = coordinates of the plane. P RMS = root square mean of distances between the plane and point Moreover, equation (5) shows the model of the circle which is created from cross section of the cylindrical segment which extracted point cloud of MLS-a and MLS-b.
R =radius of the circle Therefore, the value for evaluation for matching between point cloud of MLS-a and MLS-b are calculated with equation (6)  RMSc = root square mean of distances between centre coordinates of circles. (3) is the non-linear equation of the planar model. However the partial differentiation for each unknowns transforms the equation (3) into linear equation. Equation (7) is the partial differentiation of equation (3) with
Further on, Equation (11) is the partial differentiation of equation (3) with respect to   .
Hence, equation (14) shows Jacobian matrix for n points.

Cylindrical constraints:
The partial differentiation for each unknowns transforms the equation (5) into linear equation as same as planar one.
Equation (17) is the partial differentiation of equation (5) with respect to x .
Where 1 J = Jacobian matrix with respect to x Similarly, with respect to y , z ,  Therefor cylindrical constraints can use with the same equation of (14), (15) and (16). As consequence, the value from planers and cylinders can be substituted at the same time.

3D reconstruction with updated parameter and the revaluation process
With updated extrinsic parameters which are evaluated at the previous section 2.3 laser scanner data of MMS-b will be reconstructed with equation (1). Equation (4) and (6)

Comparison of RMS between before and after update the Coefficient of LM methods
In order to optimise the evaluation of indeterminatenesses Levenberg-Marquardt method has been applied. The algorithm of this method can get the best of the advantage of Gauss-Newton algorithm and gradient decent algorithm by modification of the parameters during execution. This parameters works as a damping factor. If the decent is faster the coefficient  changes to small value and become close to

RMS
As the termination condition the threshold  is 10 -5 .

EXPERIMENTS
In order to evaluate proposed method we carried out the experiment to calibrate with the railway environment. We used the test rail for construction vehicles at Kobe machinery maintenance center of Nikken Corporation in Kobe City of Hyogo Prefecture. The test rail composed of the straight rail and the curve rail which merged with the left-hand railroad switch. (13) The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2020, 2020 XXIV ISPRS Congress (2020 edition) Figure 3. Overview of test rail of the site  Figure 3 shows an image map of the site. Table 1 shows the specification of the MLS data acquisition. The deployment of MLS is Front and Rear. MLS run one round trip of which course includes curve and straight rails. Table 2 shows the accuracy compared with the coordinates obtained by Total Station. MLS-a has better accuracy than MLS-b as a result. Therefore the extrinsic parameter of MLS-b toward MLS-a are calculated. Following to the work flow planar and cylinder segments of point cloud are extracted from 3D point cloud of MMS-a. Figure 4 shows the example of Cylinder segmentation of the poles along the rail. Cylindrical segmentation are conducted from the ground level 0.5m of the height with 1.0m thickness and with 0.5m separation. Moreover, Figure 5 shows example of partial parts of segmentation of rail sleeper. As an initial reference value RMSP and RMSc are evaluated from segmented point clouds. Furthermore Laser scanning data of MLS-b are reconstructed to 3D point cloud with interpolated POS data of MLS-a with initial extrinsic parameter. The calibration of extrinsic parameters are conducted with the first damping factor and obtain updated extrinsic parameters. Continuously, with these extrinsic parameter 3D reconstruction of MLS-b are iteratively conducted. RMSnew are created and compared with RMSold regarding to every reference segments of planar and cylinder. Finally, damping factor  are modified according to the result of the comparison. Until  becomes less than 10 -5 these process are in loop and iteratively calculated.

RESULTS AND DISCUTION
We applied to proposed method with 34 planar and 8 cylindrical segments which were extracted from point clouds of MMS-b. Most of planar point clouds are extracted from the sleeper of the rail and the cylindrical segment are extracted from    Table 3. Extrinsic parameters and accuracy of Laser scanner use planes and cylinders constraints together the electrification poles. The calculation result shows the final process of comparison of RMS in Figure 6. The changes between the just before final and the final RMS value specified that distance from the reference segments of point cloud converged to the less value than 0.001m. The extrinsic parameter in final RMS update results are shown at Table 3.

Switch blade
Pole

Rail Courses
Update Initial

Extrinsic parameter
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B1-2020, 2020XXIV ISPRS Congress (2020 This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B1-2020-277-2020 | © Authors 2020. CC BY 4.0 License. The optimization calculation looped 5 times and took 15 minutes with Intel core i7 CPU 3.6 GHz. Therefore, the result of total calculation process achieved in practical time for the MLS data processing. We also evaluated the calibration performance qualitatively. Figure 7 shows the cross section of rails obtained by two MLSs directly above the rails. The cross section at curve rail of MLS-b which reconstructed with the POS data of MLS-a are displayed with point clouds of MLS-a. The test rail settled with the size of narrow gauge which has a separation of 1067mm without any slack and cant. The point clouds are overlaid along the rail gauge and ballast. The cross section view of the gauge appeared in same position of rail head and foot. Therefore, the proposed method could realize highreproducible positions for each rail. The simultaneous data acquisition by two MLSs directly above the rail can reconstruct the position of rails according to the accuracy of calculation for extrinsic parameter. Thus, within 0.006m of the accuracy the calibration results can be applied to the data acquisition for the railway environment. However, the proposed method remains challenges regarding to data acquisition at the switch blade. Figure 8 shows the model of point blade of switch from orthogonal and lateral direction. The structure of switch is complicated to lead the vehicle's wheel from one track to another mechanically. When the railway vehicle passing the switch rail the vibration caused by the gap of the switch rail mechanisms transmitted to MLSs. Figure 9 shows the point blade captured by Terrestrial Laser Scanner (TLS). TLS scanning were conducted from outside of the rail gauge the occlusion occurs along the rails. However point shape of the point blade are reproduced with fair quality. On the other hand the proposed method has the difficulty regarding the quality of reproduction of switch rail. Figure 10 shows the point cloud of point blade captured by the proposed method. Regarding the orthogonal view of blade rail the quality of reproduction is relatively fair. However, the lateral view shows the gaps of point clouds between the two MLSs. The gap were possibly caused by the pitch condition comparing with roll and yaw condition because apparent gaps were not found on the orthogonal view. Therefore vertical vibration effected to the calculation of 3D reconstruction of MLS-b. The Euclidean distance from POS of MLS-a to Laser Scanner of MLS-B were calculated 3.64m. The distance causes the gaps of the timing when the wheels under the MLSs ride on the blade rail. When MLS-b was about to pass on the gaps of blade rail MLS-a was still on the main rail. Therefore, the timing of vibration are different from MLS-a to MLS-b. Furthermore, while MLS-a was settled on rear cargo MLS-b was on top of the cabin of railway vehicle. The condition rigidity between POS of MLS-a and Laser scanner of MLS-b were possibly effected to the pitch element when passing the gaps of blade rail in different timing. Therefore, the improvement of the deployment and settlement should be considered using the small and low cabin type of railway vehicle. However, the advantage to use the railway vehicle is to reserve the time and cost of operation. In terms of the practical case of the railway work the time is quite limited while the absence of train operation. Although MLS equipped on the railway vehicle can start the data acquisition from the crossing the calibration site preparation are required in general cases. However, with the proposed method features along the railway can be utilized as the reference of the calibration. Thus, the proposed method can conduct fully automatically in the general environment of railway maintenance operation.
In the experiment we used only two poles and segmented to the four elevation levels at each poles. Therefore in the practical environment we should consider to the environment for calibration along hundreds meter of the rail which contains many planar and cylindrical feature. At the railway environment in the business operation many features can be utilized as planar and cylindrical features. The beam between electrification poles which relatively locates up above the railway vehicle can be The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2020, 2020 XXIV ISPRS Congress (2020 edition) This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B1-2020-277-2020 | © Authors 2020. CC BY 4.0 License. utilised as the constraint as a transversal and cylindrical features. Moreover, protection panels of railway facilities can be adapted as the planar constraint.

CONCLUDION
In this paper, we proposed a calibration method applying the mutual orientation method based on the collinear and coplanar condition of photogrammetry techniques in order to stably match the point cloud of the two MLSs directly above the rails. We present the calibration method to evaluate the relative orientation between two MLSs system units directly above the rails. The non-linear least square means were applied with LM method to take advantage of calculation time. We conduct experiments in the actual rail situation equipped with electrification pole and rails sleeper for the constraints. The accuracy of 0.006m between two laser point clouds can be achieved with this method. However, the robustness at the actual railway environment should be considered. Particularly, the vibration of vertical direction should be considered at the rail joint. Furthermore, because railway environment requires to satisfy with various condition, such as time, climate and temperature, etc. The authors are currently working on the realization of the practical application in the railway environment which actual trains are operated.