APPLICATION OF PHOTOGRAMMETRY AND IMAGE PROCESSING FOR THE STUDY OF POROUS SURFACE COURSES

This article presents an application of photogrammetry and image processing for road sample analysis. The technique makes it possible to measure the void content of a sample and at the same time to analyse this physical characteristic as a function of the depth relative to the pavement surface. In a first part, the device dedicated and data acquired will be presented, then our algorithm (pore extraction and geometry processing) will be described and finally the results will be discussed, both on the quantitative aspect and on the transmission via a dedicated report. * Corresponding author


INTRODUCTION
When traffic noise exceeds some predetermined threshold, road authorities usually replace the traditionally dense surface course by a porou s one that absorbs the acoustic waves. With an initial noise reduction of about 8 dBA, those are considered as an effective way to tackle the issue at its source. Unfortunately, the acoustic lifetime is generally below expectations since the pores are clogging within 2 to 3 years (Balmer, Steiner, 2018) (Laferrière, Beltzung, 2017). The purpose of this paper is to present a method of inspection of the clogging process to facilitate an effective maintenance procedure. The method involves the use of image analysis on pictures of impregnated cores taken from the road. Figure 1.a presents the coring machine which takes samples from the road.  First, the acquisition system and the data set will be described. Then, the steps of extracting pores by image processing as well as the geometry will be explained. Finally, the results (transmission report and comparisons with mercury intrusion porosimetry) will be presented.

Image production
To assess the state of clogging of the porous wearing courses, cores of diameter 150 mm are cut from the site. The porous layer is sawn from the rest of the core and sealed along the curved surface while the two flats remain accessible. An appropriate epoxy resin is coloured in fluorescent red or yellow. The porous layer is fully vacuum impregnated. Firstly, the top of the layer (wearing surface) is filled with one of the coloured resins. After sufficient hardening of the latter, the specimen is turned up-side down and the remaining air voids are then filled with the second resin. Finally, the specimen is sawn into two pieces to obtain a pair of sections. Each section is photographed under UV light to reveal the UV fluorescent material filling the pores.

Shooting device
The Figure 2 shows the photographic set-up. The shooting device consists of:  a support for the camera (18 MPixels (5184 x 3456) and a camera lens of 40mm).  a support for placing the specimen with stops allowing it to be repositioned always in the same place  an ultraviolet lamp for rendering the fluorescent resin There are many advantages of this system: parameters of camera (aperture, exposure time, ISO) are constant and can be set once and for all.

Data
The Figure 3 presents two pictures taken by the shooting device. One can clearly see the upper pores filled from above with an orange resin, and the lower pores filled from below with a yellow resin. The zone between the 2 resins indicates the clogging front: the closer this empirical line is to the road surface, the less efficient is the acoustic effect because there are no longer enough accessible pores to absorb the sound waves. The specimen presented on Figure 3a shows a new porous wearing course: the orange resin has penetrated the entire sample. The specimen presented on Figure 3b shows an old porous wearing course: the orange resin has been blocked by soiling. Once the image is acquired, automatic data processing can take place. The different steps are:  Extraction of pores (see section 3)  Levelling the specimen relative to the road surface and scaling to make metric measurements (see section 4)  Computation of characteristics and data enhancement (see section 5)

AUTOMATIC PORE EXTRACTION
Once the image is acquired, automatic data processing can take place. The first step consists to extract pores. Figure 4 shows the different cases we have. Figure 4: Examples of pores filled by resin: (a) resin on the top, which is "yellow" in surface and green in the sample. (b) isolated soiling cluster in a pore. (c) mix between the 2 resins.
(d) orange which is less variable in colour than yellow.
To extract pores, we have chosen a change of space colour from RGB (Red Green Blue) to HSV (Hue Saturation Value), as explained on Figure 5. We must determine the appropriate quantities for the thresholding. To do this we have selected a few pores (less than 5 in practice) and then calculated the shade levels for the two resins. The saturation levels and values are of little importance in the grading process: bitumen and aggregates being very dark because they are not fluorescent. Finally, for each pixel of the image, the algorithm checks if they have the same characteristics as the yellow or the orange resins. The Figure 6 presents the result of the treatment: the blue stains correspond to the pores communicating with the road surface while the orange stains correspond to the pores communicating with the bottom of the porous layer. The area between the two colours indicates the clogging front. This is an all automatic process that takes only a few seconds.

Camera calibration and scaling
In order to make metric measurements in the images, it is necessary to determine the intrinsic and extrinsic characteristics of our device. The positions and orientations of the specimen and the camera are mechanically constrained by stops: there is no play and the camera is facing the specimen. The distance between the camera and the specimen is 20 cm and the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) positioning deviations are approximately 1 mm, hence positioning errors introduce a scale change of the order of 0.5%. Concerning the distortion, we could completely measure it and model it by using the fundamentals of photogrammetry thanks to software like MicMac (Deseilligny, 2011). We first chose to check the influence of the distortion in our device. To do so we photographed a checkerboard with boxes of 1cm per side, then measured the area of the squares at different places in the image by applying a single scale factor to go from a pixel measurement to ground measurement (250 pixels  1 cm in our case). The Figure 7 presents the image of the test pattern used to measure the distortion.

Figure 7: Checkerboard photographed by the device
Not taking the distortion into account leads to an error of the order of 1%, which is negligible in the context of this study (Table 1).
Min (cm 2 ) Max (cm 2 ) Mean (cm 2 ) Std Dev (cm 2 ) -0.016 0.004 -0.003 0.006 Table 1: Deviations between the actual measurement (1 cm 2 ) and the measurement value The main advantage of not taking the distortion into account is to facilitate calculations to move from image geometry to field geometry (no resampling).

Levelling specimen in the surface pavement
The geometry of the specimen consists in determining the average slope of the top surface of the core, then turning the image to remove the slope. To do so we use the algorithm described by steps below and illustrated in Figure 8. a: extraction of the road surface b: threshold of these contour c: extraction for each threshold d: elimination of extrema points e: calculation of the slope of the line passing by the remaining points f: rotation of the picture The Figure 9 shows the picture before (a) and after (b) rotation. After defining the right geometry, we can carry out the image processing and extract all the characteristics

Comparison with mercury intrusion porosimetry and bulk density by dimensions
In order to validate the measurements, we have compared our method with 2 other methods. The first is the mercury intrusion porosimetry. This technique consists of measuring the porosity of a material by applying controlled pressure to a specimen immersed in mercury. Specimens for porosimetry and image processing have been taken from the same core. The second method is the determination of bulk density by measuring the dimensions and the mass of the specimen. From this value and the maximum density of the bituminous mixture the void content is calculated according to European Standard EN 12697-8. Based on a series of 5 samples, the absolute difference in total porosity between these 3 techniques is in each case less than 2 vol.-% ( Table 2). Remember that the specimen is sawn into two pieces to obtain a pair of sections, corresponding to Image 1 and Image 2 in Table 2. From this double analysis we infer that the reproducibility of the method is satisfactory. The main advantage of our process is that we can compute more characteristics as shown in Figure 10 and as explained in the next section. Figure 10: Example of an automatic image analysis with indication of the total porosity, the pore size distribution and the profile of total and clogged porosity as a function of depth (the core diameter is 150 mm).

Pore analysis
Once the pores have been extracted, the characteristics of the sample can be calculated and formatted with the aim of analysing the state of clogging of the wearing course. The results of the analysis are presented in Figure 10. We can compute, by counting pixels, the following characteristics:  global porosity  distribution of the pore sizes  profiles as a function of depth of total and clogged pores 6. CONCLUSION AND PERSPECTIVE

Conclusion
Application of photogrammetry and image processing for the study of porous surface courses has greatly contributed to a better understanding of this type of bituminous mixtures. The technique provides the opportunity to analyse both the geometry and the size distribution of the pores in the specimen. Further, by taking cores over time, we are able to follow the process of clogging and monitor the outcome of an acoustic maintenance for instance by grinding the road surface (Beltzung, Balmer 2020). The accuracy of this approach by image analysis has been robustly validated by comparing the total porosity with two other methods whose principles of measurement are fundamentally different. Furthermore, the method is reliable since two surfaces from the same specimen provide very similar values.

Perspectives
From a road maintenance point of view, the next step will be to find a correlation between the pore size distribution and the kinetic of clogging. From a processing point of view we seek to improve the automation in order to be able to analyse several hundred images to study the acoustic behaviour of road surfaces over time.