ENVIRONMENTAL DATA DELIVERY FOR AUTOMOTIVE SIMULATIONS BY LASER SCANNING

: The development of autonomous vehicles nowadays is attractive, but a resource-intensive procedure. It requires huge time and money efforts. The different carmakers have therefore common struggles of involving cheaper, faster and accurate computer-based tools, among them the simulators. Automotive simulations expect reality information, where the recent data collection techniques have excellent contribution possibilities. Accordingly, the paper has a focus on the use of mobile laser scanning data in supporting automotive simulators. There was created a pilot site around the university campus, which is a road network with very diverse neighborhood. The data acquisition was conducted by a Leica Pegasus Two mobile mapping system. The achieved point clouds and imagery were submitted to extract road axes, road borders, but also lane borders and lane markings. By this evaluation, the OpenDRIVE representation was built, which is directly transferrable into various simulators. Based on the roads’ geometric description, a standardized pavement surface model was created in OpenCRG format. CRG is a Curved Regular Grid, containing all surface height information and objects, but also anomalies. The 3D laser point clouds could easily be transformed into voxel models, then these models can be projected onto two vertical roadside grids (ribbons), which are practically an extension to the OpenCRG model. Adequate visualizations demonstrate the obtained results.


INTRODUCTION
The development of autonomous vehicles is one of the most perspective investigations nowadays.It covers wide areas of applied research, where the geospatial technologies can also play a significant role.Because the developments in the automotive world are extremely expensive and time-consuming, all techniques which can reduce these drawbacks are welcome.This recognition has led to the advent of simulations, where welldefined virtual environments are created and the vehicles, as well as their functional components, can be tested carefully in controlled circumstances.The simulators are modeling tools, based on automotive and environmental/infrastructural virtual models.The model simplifies reality and supports a wide spectrum of simulations for the autonomous driving system such as vehicle dynamics and steering simulation, complex traffic scenarios, etc.If these models are too simple and not adequately realistic, the results of the simulations can lead to erroneous behavior of the vehicle.To avoid this weakness, the best way is to digitize the infrastructure and the neighborhood as accurate as possible.The state-of-the-art surveying techniques, like mobile mapping, has the expected capabilities to capture the required details with acceptable data quality.These systems have suitable economic features as well: the speed of surveying, the interaction demand against human operators.Laser scanning technologies have productivity nowadays to capture environmental details with sufficient resolution (Marshall et al. 2016) (Wang et al. 2019).Thanks to the simultaneous image capture, the obtained laser point clouds can be colorized, so the interpretation of the terrain features can involve also object color information.With evaluating the acquired field data, any level of details can be achieved, which is portable to simulator software.The current paper aims to demonstrate the potential of laser scanned data in the use of automotive simulations.To show an overview, the following flowchart has been created (Figure 1).

Figure 1. Flowchart of laser scanning data processing
The paper is organized as follows: Section 2 presents the applied laser scanning technologies and the pilot sites.Section 3 gives a simplified overview of the simulations and tools in the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W16, 2019 PIA19+MRSS19 -Photogrammetric Image Analysis & Munich Remote Sensing Symposium, 18-20 September 2019, Munich, Germany automotive world.Section 4 describes the applied methodologies and the obtained results can be read in Section 5. Finally, a conclusion section and references can be found.

LASER SCANNING AND PILOT SITE
The survey of the road network requires efficient technologies because a lot of details has to be captured with high accuracy and the environment has a permanent change.Laser scanning is an excellent technology to fulfill these expectations.Terrestrial laser scanning (TLS) is a technology, where a static position is chosen and the scanner acquires point clouds and images about its surrounding.TLS has a very impressive geometric resolution but limited capacity in road survey.We have applied TLS technology therefore only for validating the mobile mapping.The used instrument was a Faro Focus 3D 120S.The scanner has a maximum measurement range of 120 m; its maximum data capture frequency was 976 000 points/sec.The horizontal view angle is 360°, while the vertical one is 305°.The scanning resolution was set for 6 mm at 10 m distance.A laser scanner installed on the mobile mapping platform is called mobile laser scanning system (MLS).We could have access to a Leica Pegasus Two system, which is equipped by a Novatel GNSS (Global Navigation Satellite System) and IMU (Inertial Measurement Unit) positioning system, a Z+F profile scanner and seven color cameras.The laser scanner has a maximum range of 120 m, the rotation speed of the mirror is 200 rps.The laser beam has a wavelength of 635 nm.The viewing angle is 360°, the distance resolution is about 0.1 mm.The data acquisition was planned in the neighborhood of the Budapest University of Technology and Economics (BME).The field survey was conducted on 18 January 2019, when the weather was cloudy and slightly sunny.The traffic on this Friday was moderate, it has not disturbed the continuous data capture.The trajectory of the surveying can be seen in Figure 2. To achieve more accurate survey results, the mobile mapping system was extended by a base station, which is a permanent GNSS station on the BME's roof.The average absolute accuracy of the trajectory was 4.9 cm, while 3.9, 1.8 and 1.3 cm in X, Y and Z directions respectively.
The cameras have a geometric resolution of 2046 × 2049 pixels, the radiometric resolution was 24 bit.The lenses have a focal length of 8.0 mm, except the only zenith camera with 2.7 mm.The image capture was made with 8 fps; the used storage format was JPG.
The measurement was split into 3 parts in order to achieve better data management.The total surveyed road length was 4571 m with 951 captured color images.The obtained 25 laser scanning point clouds have 13.7 GB.

AUTOMOTIVE SIMULATIONS AND TOOLS
The The second group is the driving simulators, which contains not only the entertainment packages but also the training and the research and development software tools.By the help of these packages, the engineers can study the driving characteristics of the vehicles, their reactions on the various road and other environmental parameters.Representative tools are TASS Prescan (TASS 2019), Vires.The developing company of Vires has been elaborated a standard family with OpenDRIVE, OpenCRG, and OpenSCENARIO, which has become an important format for automotive developers and firms.Driving simulators can also be found as part of MathWorks Matlab: a dedicated Driving Scenario Designer is a tool for similar analyses (Figure 3).The developed method is based on laser scanned point clouds.In order to ease and accelerate the model building as well as some preliminary visualization will be available, a voxel model for the area was created.
The point clouds were originally in LAS-format.Our application reads the LAS-files and corresponding road axis lines.A cut-off with 50 m wide buffer zone is created by LAStools (rapidlasso 2019), then the obtained point cloud was converted into PCDformat by CloudCompare (CloudCompare 2019) function calls.
In the same step, a coordinate shift is executed and the shift parameters are also stored.After preparing all affected original point clouds, the temporary created PCD-files are imported one by one, where the point clouds were transformed into the road axis' local coordinate system.By merging all point clouds, the voxels are created and their perpendicular distances are computed and stored.The algorithm finishes the run by saving the voxel model in OBJ-format.Since the voxels have the perpendicular distances from the road axis, their projection onto the left and right side ribbons are very fast.The two obtained depth ribbons are stored as images.
The application was developed in Visual C++ in Windows.The image management has been supported by the OpenCV library (OpenCV 2019), while the point cloud handling by Point Cloud Library (PointCloudLibrary 2019).

RESULTS
The OpenCRG model is a regular grid with different visualization options.The grid was created on a road segment near to the BME campus, where the pavement has been built by cobblestones (Figure 5).These big cubes have typical gaps forming a pattern on the surface.If a complete stone or a part of it fails, a pothole appears, which increases during the time.
Potholes decrease the driving comfort or can be dangerous, so their presence must be known by the human or autonomous drivers.The manholes are usually made of steel, which has different roughness and slipping characteristics than the pavement.They are also important surface elements.The highresolution model of OpenCRG contains all these elements.The presented example has 715 × 201 elements with 1 cm geometric resolution.The projection of the voxel model onto the vertical ribbons is nothing more than creating orthogonal views of the road neighborhood.Because the voxels were filled with distance information, these ribbons are depth orthoimages (Figure 7).The derived left and right side vertical ribbons can be combined into "street view" visualizations; as it can be seen in Figure 8.
Because the ribbons inherit the original reference system of the laser scanning similarly to the OpenCRG model, they are also easy to be combined with further GIS contents, like OSM layers.
Figure 8.A "street view"-like visualization of both sides

CONCLUSIONS AND OUTLOOK
The mobile laser scanning technology has been found as an efficient and accurate data acquisition methodology for road network surveying.It has the geometric resolution which fulfills the expectations of the highly automated driving.The different simulators of the nowadays automotive world must be fed by data captured and evaluated from reality.Instead of unique data transfer solutions, the standardized formats have to be preferred.The evaluation of the mobile laser scanned point clouds and the corresponding color imagery gives the possibility to extract the axis and borders of the roads, furthermore also the lane borders and lane markings.In a suitable format, this information can be used in GIS and navigation applications.
If the relevant road geometric data is available, the OpenDRIVE representationa standardized road description known by the automotive simulatorsis also possible.The carmakers and their developers can profit from the geoinformation data sets.
With the precise road geometry and the high-resolution laser scanned point clouds, OpenCRG models representing the road surface can be produced.The Curved Regular Grid describes the exact vertical features of the road (longitudinal and cross sections), but also the pavement characteristics, e.g.roughness circumstances.The OpenCRG model covers furthermore the objects of the roads, like manholes and anomalies like gaps or potholes.The vehicle suspension modeling procedure can directly apply the provided data sets.
If the 3D point clouds are converted into voxel models, auxiliary visualization and analysis options have been opened.Because the derived voxel models are georeferenced, their integrative usage is natural.Again, GIS and navigation systems can be the targeted application areas.
The voxel models obtained along roads are ready to be transferred into projected visualizations.When a virtual roadside vertical ribbon pair is created, the road neighborhood with buildings, road furniture, fences, and vegetation can be projected onto these ribbons.Since the so formed ribbons are grids, like images, their management is not a strange task.The paper brought an example to demonstrate depth images in street view like visualization.Naturally, the voxel models can contain not only the perpendicular distances but the color information taken from the laser scanning cameras, a complete roadside orthoimage series can be achieved by this technology.
The paper has shown the possibilities hidden in the laser scanned point clouds.The practical works still require additional developments, especially in the automatization and data storage.
Obviously, the integration with the automotive simulations may also be improved in the future.

Figure 2 .
Figure 2. The mobile mapping trajectory in the neighborhood of the BME.The only road with cobblestones can be found in point A development of vehicles requires an extreme amount of time and money.Nowadays the development procedure can effectively be accelerated and the costs can efficiently be reduced by the use of automotive simulators.The design of a vehicle chassis has long been supported by Computer-Aided Design software packages.The design of a vehicle chassis has been supported by Computer-Aided Design software packages for a long time.Newly not only the outlook but the "interior" can also be developed by computers.Beyond the mechanical construction packages, several testing environments also arise.The automotive simulators can be grouped into three main categories: vehicular testing applications, driving simulators and traffic simulators.The first group is for simulating and testing the mechanical behavior of motorbikes, passenger cars, buses, and trucks.These software tools are involved in analyzing vehicle dynamics, calculating performance characteristics, developing active controllers, studying crash behavior and many more.Representative packages are among others AVL (Advanced Simulation Technologies 2019), BikeSim, CarSim, TruckSim, but also SuspensionSim (Mechanical Simulation Corporation 2019), dSPACE (dSpace 2019), IPG Carmaker, IPG TruckMaker, IPG MotorcycleMaker (IPG 2019).
Figure 3. Driving Scenario Designer in Matlab with road network definition in OpenDRIVE

Figure 4 .
Figure 4.The coordinate system of the road with the corresponding grids of the left and right side

Figure 5 .
Figure 5. OpenCRG model of a road segment.Note that the cobblestones (A), pothole (B) and manhole (C) can be recognized During the processing workflow of the road neighborhood, the obtained 3D voxel model inherits the georeferencing information from the original point clouds.This feature makes it easier to use the model together with other GIS data sources like Figure 6 demonstrates the voxels with OpenStreetMap (OSM) layers.OSM was limited to the buildings, roads and green areas.The presented voxel model contains 78392 voxels with a voxel size of 1 m.Because each voxel is stored as float values containing the distances, the model requires ~25 megabytes on the storage in ASCII format OBJ-file.The model was built in about ~40 seconds on an average laptop.

Figure 6 .
Figure 6.The voxel model of the main building of BME together with OpenStreetMap layers

Figure 7 .
Figure 7. Orthogonal view of a right side depth ribbonThe above cutoff was created with a cell size of 10 cm.The grid has 3420 × 430 pixels (means 342 × 43 m).Because the geometric resolution is a freely adjustable parameter, the ribbons can have any arbitrary cell size.