RPAS BASED TRACKING OF MACHINERY USED IN ASPHALT PAVING PROCESS

Application of rigorous construction procedures is fundamental when it comes to road construction. Nowadays, and in particular in Chile, the asphalt paving construction process is based on tradition, craftsmanship and other implicit, experience-based methods employed during road construction. This means, that the different stages of the construction process, including the paving of the road, are not controlled on time as they should, which causes that the final product does not have the ideal characteristics for which the construction materials initially were designed and manufactured. Therefore, we propose the use of tools and methodologies that allow monitoring each of the stages of the construction process in almost real-time. Our approach aims to evaluate the use of Remotely Piloted Aircraft Systems (RPAS) during the asphalt pavement construction process by capturing images and videos during the placement of the asphalt mixture and the movement of the compaction machinery (pneumatic roller and drum roller) present in the construction site. The observations are digitally processed by mainly binarization and thresholding. As a result, the macro texture of the asphalt surface was estimated and the trajectory of the compaction rollers was visualized. In conclusion, our results reveal that the main factor that influences the quality of the roads corresponds to the performance and operations of the compaction machinery. * Corresponding author


INTRODUCTION
The asphalt pavement construction process consists of several steps, which must be understood quite well and carried out in such a manner that the final product complies with national and international standards. Until now, a lot of investigation has been carried out to determine the perfect mixture, (Yu et al., 2017), (Sanchez et al., 2017), (Das et al., 2015), (Wang et al., 2018), of the raw material used for paving. Comparing scientific articles published over the past decades, papers on asphalt material and mix design significantly outweigh the number of papers published with regard to asphalt pavement construction. Nevertheless, when it comes to the paving process itself, most of the work is done in an intuitive and poorly-controlled manner (Bijleveld et al., 2015). In particular, there is no real-time, image-based monitoring carried out, which would permit to know whether the construction process is carried out properly or not. The asphalt paving construction process has been based for a long time on tradition, craftsmanship and a variety of implicit, experiencebased methods employed during construction activities by the personnel itself. In such a construction model, many of the construction activities carried out frequently cannot be controlled or improved even if it is necessary. This means, considering the high cost to society of road construction in general and asphalt concrete pavements in particular, a scientific approach has to be achieved. For example, the execution and operation of the machinery (paver screed, vibratory steel wheel roller, pneumatic tire rollers, etc.) used during the installation of the new asphalt surface has to be monitored with extreme rigor as compaction is of great importance in order to obtain a high-quality pavement (Linden et al., 1989). Sometimes, when looking at a cross section of an asphalt road, its compaction in the center is higher than at the edges (Krishnamurthy et al., 1998). Compaction is affected by several variables such subgrade support, working temperature, material properties, outside temperature, layer thickness, aggregate grading, the mix design, construction equipment and variables such as compactor speed, compactor frequency, amplitude, wheel load and, of course, the number of passes performed by the compactor operator (Miller et al., 2007). It is up to him to take decisions about time, speed, trajectory and whether to apply vibration or not. Unfortunately, and as this is done in a mostly intuitive way, inhomogeneous compaction is the result. This effect is caused as the operators tend to make more passes in the center of the road -unions and edges are less compact. As a consequence, this results in most of the cases in excessive or insufficient compaction, depending on the number of passes over the same area. Irregularities during compaction process frequently cause defects in the final structure of the pavement, (Bouvet et al., 2001). These errors are due to limited human capacity and rationality. The complexity of the road roller operators' tasks negatively affects the quality, productivity and project cost. Consequently, knowledge of the explicit process can provide support and a deeper understanding of the process being followed. An enhanced control over compaction processes provides significant improvement in pavement quality. As stated out in literature (Pradena et al., 2020), in the late twentieth century, new technologies were barely used in previous studies, which aimed to determine compaction variabilities (Leech et al., 1976, Roberts et al. 1996. Nevertheless, they detected deficiencies in compaction mainly caused by the complexity of the process. One possibility that was explored (Bouvet et al., 2001, Oloufa, 2002, Pampagnin et al., 1998) aimed to incorporate real-time GPS positioning as an assist system for the operators. The final goal was to implement a method based compaction strategy which would allow reducing complexity of the compaction process. Based on GPS compaction prototypes, which were developed afterwards, today's roller compactors are typically equipped with intelligent compaction applications (Peyret et al., 2000 (2), Anderegg and Kauffmann, 2004), which include not only GPS receivers and antennas but also other sensors to assist the roller operators in real-time. Although nowadays several compaction technologies are available, its adoption in asphalt construction has been slow as the construction industry has a couple of doubts (Bowden et al., 2006, Panagiotis andTatum, 1999). One very important argument is that the technology is far too complex for typical construction workers who in general are not well trained to understand and manage such technology.
Being able to control and monitor the compaction process is fundamental to achieve the requirements of the paving project and to comply with established standards and norms by national and international institutions. The use of RPAS (Remotely Piloted Aircraft System) for this type of civil engineering activities appears in response to the need to monitor and capture images that allow visualizing the progress of the construction stages and detecting opportunities that provide improvements during the construction process. In recent years, the use of RPAS to obtain images and therefore their application in the field of civil engineering has experienced remarkable growth. Although this technology generates great interest its final use still presents difficulties, because it is necessary to process the observations to obtain relevant and useful information.

METHODS
In our approach, we propose a RPAS based monitoring with subsequent digital image processing to be carried out during the installation of new asphalt surface. Emphasis is placed on an image-based analysis of the trajectory of the involved machinery, mainly vibratory steel wheel roller, pneumatic tire rollers, as it is essential to determine surface homogeneity and the negative consequences of mix segregation and voids. This means, that during the asphalt paving process itself we capture videos in real time of the asphalt paving and compaction process, with a RPAS (in the present study a DJI Phantom 3 Pro, equipped with a 1/2.3¨ CMOS sensor camera with 12.4M effective pixels), which we flew at an approximate height of 150 meters (figure 1). Additionally, we captured still images at an approximate flight height of 6 meters, which allows us to get a more detailed view of the asphalt mix placed during the construction process and its degree of compaction as the machinery advances.    In order to detect the magenta plates, and in consequence the machinery involved in the construction process, a set of color range intervals are established for each of the bands of the RGB image. This way, we were able to classify each of the pixels of the image by means of an iterative process. In case that the pixel values coincide within the three determined intervals, the pixel was declared part of the magenta plate and the value 1 was assigned to that pixel. In any other case 0 was assigned ( figure  3). As a result we obtained a binary image. That way, it is possible to determine the position of the machinery. In case of the still images, our approach is quite similar to the already mentioned one. Again, we defined color range intervals but this time to detect visual differences in the asphalt surface due to the back and forth movement of the machinery during the compaction process (how the stone objects within the asphalt mix change their position and orientation during the process).

RESULTS AND DISCUSSION
The visual records captured with the drone, reveal how compaction process is carried out. Although there were a few inconveniences (such as strong winds, a non-stable platform, irregular passes and movements of the paver and rollers), during the observation, several interesting results were obtained.
At first, the trajectory of the involved machinery mentioned earlier during the paving process and how they evolved over time were charted, thanks to the detection of the magenta plates.
As an example, figure 4 shows the passes made by the operator of the pneumatic roller. One important observation that can be made reviewing the trajectory is that our suspicion of nonhomogenous passes was confirmed. There is no record in the upper part of the graph, which means that the operator did not consider compaction at the edge of the road. Furthermore, close to the center of the road are multiple registries and therefore compaction in this area is higher. And finally, the operator did not reach the same start and end point at every trajectory either. So, the compaction area extension in direction of the paving progress is not unique.
Furthermore, as the passes as expected did not fit a regular pattern, visible deficiencies in the asphalt surface could be detected. This means, that that the road is not receiving a uniform and equal compaction in each section of the road (figure 5).

Figure 5. Result of irregular passes of the machinery (true colour imageleft; binary imageright)
A fact that could be confirmed in the calculated binary image (highlighted in figure 5 by a red circle). The respective pixels of that highlighted area were not detected as compacted and therefore a pixel value (0) that indicates not compacted was assigned. image. They represent the shadows that turn out between the stone objects within the asphalt mix when this is placed. When the compaction process ends, the situation is different ( figure 7). The three images on the right (figure 7), correspond to the original image captured from the drone, after the compacting machines carried out their work. The images on the left ( figure 7) show the result of the digital image processing. The yellow pixels represent the darkest pixels (DN close to zero) in the original image. Again, they represent the shadows that turn out between the stone objects.

Figure 7. Final distribution of stone objects
Other aspects, such as color, texture and orientation can also be analyzed.

CONCLUSIONS
Our approach provides evidence that there are opportunities for improvement during the road construction process and, in particular, the compaction stage of asphalt paving. The visual records captured with the drone, explicitly show the execution management by the operators of the compaction machinery. Our findings indicate that the trajectories of the monitored machinery used during compaction, is not regular and that there are chances for improvement. As mentioned earlier, there is no established, or normalized, strategy which is used to carry out the compaction process. Hence, our results confirm the findings presented by (Pradena et. al, 2020), which clearly indicate that the number of passes and trajectories are not carried out in a uniform manner. As a consequence, the compaction strategy is not optimal during construction. The information obtained by this research can be used by specialists in the construction area for the design and development of new methodologies that solve the problem of irregular passes of the compacting rollers over the asphalt surface to improve the final road quality. Therefore, and in future work, from our point of view, it is necessary to establish and to carry out a (near) real-time monitoring as well as post-processing analysis of the overall process. This information can be used by specialists in the construction area for the design and development of new compaction strategies that can reduce the variability of the process and their effect on the asphalt pavement (variability in the final density of the asphalt layer), thus improving the final road quality.
Regarding the use of RPAS, on the one hand, the advantages that arise with the use of a RPAS during the construction process of a road work are the possibility of monitoring the execution of the machinery which participates in the road construction and generating visual records which can be reviewed afterwards to carry out further studies and analysis. One objective might be to correct and improve the construction process and, in consequence, the final quality of a road. And on the other hand, the disadvantages that emerge with the use of a RPAS in this particular application are related to the low flight autonomy that at present these systems have.
Considering battery and storage performance lots of work already has to be done. Although one possible solution might be to carry additional batteries in order to be able to replace them every certain time, the disadvantage will be that it would not be possible to make a complete monitoring of the whole complete compaction work. The weather and wind conditions, which play a fundamental role in the feasibility of using the RPAS, also are an important aspect. The platform has to be stable all time in order to obtain accurate and clear captures of the scene. This will not always be the case.
Considering all the aforementioned aspects, the use of RPAS in the construction process of asphalt pavement road works is currently viable, although several considerations have to be made. Taking into account the technical characteristics that RPAS have today, it is only necessary to continue improving the development of their technology so that they permit to obtain more reliable observations in terms of precision, flight stability and flight autonomy. Therefore, it is not ruled out that in the future, these platforms and their implemented technology will give very good results and will become an efficient tool for construction work, essential at each stage of construction.
To finally conclude, one interesting idea for future work can be derived from the fact that today GPS allows tracking trajectories of construction machinery. Nevertheless it is limited to the possibility to observe only the machinery and not the whole construction site. And therefore, the combination of RPAS and GPS would result in an attractive technical approach for decision makers, engineers and operators, which might improve several construction steps and achieve better results at the same time.