LOW-COST UAS PHOTOGRAMMETRY FOR ROAD INFRASTRUCTURES’ INSPECTION

All over the world, road infrastructures are getting closer to their life cycle and need to be constantly inspected: a consistent number of bridges are structurally deficient, and the risk of collapse can no longer be excluded. In contrast with the past, the interest in structure durability has recently grown rapidly. In order to make bridges durable, it is necessary to carry out ordinary maintenance, preceded by inspection activities, which can be traditionally divided in two categories: destructive and non-destructive (NDT). All the NDT inspections (visual, IR thermography, GPR) can be conducted by using UAS (Unmanned Aerial Systems), a technology that makes bridges inspections quicker, cheaper, objective and repeatable. This study presents the visual inspection and survey of two bridges by using a UAS DJI Mavic 2 Pro, equipped with a 20Mpixel Hasselblad camera that records 60fps 4K video and a 10bit radiometric resolution. Starting from the acquired data, a 3D model of each structure was built by using Structure from Motion (SfM) principles and software. To validate the two models, each of them characterized by a centimetric accuracy, the UAS camera generated cloud of points and was co-registered with the point cloud of a terrestrial laser-scanner using Ground Control Points (GCPs). To make this, CloudCompare comparison software was used; the plugin M3C2 automatically calculates the distance between the points of two compared clouds. Finally, some general rules concerning the UAS main characteristics for inspection of bridges and software for data processing are proposed.


INTRODUCTION
Both in Italy and worldwide, road infrastructures are getting closer to the end of their life cycle: an increasing number of bridges are structurally deficient, and the risk of collapse can no longer be excluded. The Morandi Bridge collapse (Genoa, August 2018) just represents one case of a broader delicate situation involving several bridges, not only in Italy but all over the world. Most of them were built when builders' main focus was the strength of the structure; recently, thanks to engineers' improved knowledge and the bridges gradual degradation of bridges, interest in structure durability has grown rapidly. Such a constructive criterion focuses on the ability of the bridge to maintain intact its characteristics during its life cycle. In order to reach this goal, it is necessary to carry out ordinary maintenance, consisting in periodic interventions, scheduled by the local authority, aimed to avoid the degradation causes of most common road. A recent census by -the National Autonomous Roads Corporation (ANAS) (Mannella, 2019), conducted on a sample comprising 3,000 bridges, highlights that more than 50% of them were built more than 40 years ago. As previously said, this situation affects bridges worldwide: a report by American Road & Transportation Builder Association (ARTBA, 2019) points out that in the US 47,000 structurally deficient bridges are crossed every day 178 million times. According to this report, 40% of US bridges need to be replaced or repaired due to their poor condition. Canada is also facing a similar issue: 40% of their bridges are either in fair, poor or very poor conditions (Canadian Infrastructure Report Card, 2019). Likewise in Europe, the French Government asked the Senate to investigate on the status * Corresponding author of French bridges. The report "Sécurité des ponts: éviter un drame" (Chaize, Dagbert, 2019) highlights that around 25,000 French bridges (10% of the total) lie in very poor structural conditions. Given this number, the authors of the report recommended the allocation of €120 million per year to repair those structures. Finally, according to an online German newspaper "thelocal.de" (thelocal, 2018), a survey conducted by the Federal Highway Research Institute (BASt) concluded that only 12.5% of Germany's motorway bridges were in good conditions, while 12.4% were in poor condition.
As highlighted, since the bridges condition became a major issue, frequent and furthered inspection activities are needed. Current bridges inspections can be divided into two categories: destructive techniques and non-destructive testing, also known as NDT. The main difference between the two is the fact that the former implicate a little structural damage: the most common destructive techniques are coring test, pull-out method and Windsor probe test. All the other methods, such as visual inspection, Infrared Thermography and GPR, are classified as NDT. All these techniques are linked by a series of limitations that have led many infrastructures managers to postpone ordinary inspection activities, or not to perform them at all, just like the current situation of bridges highlights. Major shortcomings of traditional inspection techniques are represented by their costs, duration, poor repeatability and strong subjectivity in the results. Modern technologies such as Unmanned Aerial Systems (UAS, commonly known as drones) can fix them all. By using UAS, the inspection of road infrastructures becomes quicker and cheaper, its outcomes objective -since they are independent of inspectors' personal experiences -and, above all, they become repeatable. Accordingly, UAS path can be memorized and repeated for successive inspections, so that data are collected every time from the very same positions with respect to the bridge, enhancing their comparability.

BRIDGES INSPECTIONS BY UAS
In recent times, national organizations and universities have investigated the possibility of using UAS technologies to conduct bridges visual inspections. These studies or consolidated practices are very heterogeneous and point out the extreme ductility of drones when speaking about road infrastructures inspections. Speaking about consolidated practices, starting from 2013 RFI, a major Italian railway company, has used UAS to conduct inspections of specific geometric shapes bridges or bridges that are in morphologically complicated areas (Tisalvi, Vecchi, 2019). In this way, they can speed inspection procedures by avoiding both the bridge closure to traffic and the use of heavy equipment (also known as by-bridge equipment). The main outcome of their activity is visual data (pictures or videos) that allow to enrich the Bridge Management System (BMS). Relying on these data, RFI has created bridges 3D models and geometric reliefs. Similar tests have been conducted by several Universities. A possible combination between UAS images and automated algorithm for crack detection (a major cause of RC bridges deterioration) is proposed by (Yeum, Dyke, 2015). The authors' aim is to process the pictures collected by UAS using an algorithm that automatically detects cracks on the structure, reducing both the time of the data analysis and the outcomes subjectivity. The authors developed an object detection algorithm, whose use represents the most critical phase of the entire test. Indeed, each picture differs from the others by its brightness, scale and background, making the algorithm task very complex. To detect possible cracks, the authors applied a sliding window technique, that uses a fixed rectangular window sliding on the images to decide whether they contain an object or not. This phase is called by the authors "object detection", and it is followed by two other phases: "object grouping" and "crack damage detection", that consists in the last step of the proposed procedure and allows to decide which detected objects are cracked and which ones are not. It is evident how this procedure, besides making the inspection faster using the drone, also makes the data analysis faster. Another relevant proposal to inspect bridges using UAS comes from (Omar, Nehdi, 2017). The authors of the study highlight the possible combination between UAS and Infrared Thermography, a major NDT technology for bridges inspection. Their main goal was to detect subsurface delamination in concrete bridge decks taking advantage of a particular behaviour of delaminated areas, that warm up at a higher rate than the surrounding sound concrete during daytime (vice versa during night-time). In this way, the IR images are characterized by a series of hotspots where delamination occurs. To acquire appropriate thermal images, it is essential to use a high-resolution thermal camera: its spectral range, spatial resolution, frame rate and temperature range are crucial parameters. GCP (Ground Control Points) that are visible in the infrared spectrum are also essential to distinguish each portion of the bridge deck, as well as RGB images of the same portion of the bridge deck, to avoid false positives. Starting from the thermal images acquired by the drone, the authors utilized a k-means clustering technique to identify temperature thresholds, realizing a heat map which different colours indicate different levels of delamination severity.

BACKGROUND
To conduct a proper bridge inspection by using UAS, it is fundamental to know the basic properties of the drone. First, it is essential to know that an UAS consists of three main components: the aircraft (or Unmanned Aerial Vehicle, UAV), a Ground Control Station (GCS) and a remote control, that allows the pilot to control the movement of the drone. GCS is a software installed on a PC (or a tablet or a smartphone) that communicates with the aircraft using a wireless telemetry and shows to the remote pilot all the drones-related data (e.g. its position and performance). Thanks to the GCS and the GPS device, a drone can fly autonomously following a pre-set path. Modern UAS can also condensate GCS and remote control into a single device. Multicopters are suitable UAS for infrastructures inspections: the helicopter-style structure allows them to fly vertically, to work in very tight spaces, carrying more than one instrument on their body, and especially the possibility to conduct stationary flights. This characteristic is essential to take focused images, not affected by the movement of the drone. Fixed-wings drones, on the contrary, can be used to conduct an overall inspection of the structure and its morphological context, simply by taking nadiral images. Instead, a crucial aspect of multicopters is the rotation of the camera: the greater the rotation angle, the widest the covered area into the acquired optical data. This is the reason why many drones have a gimbal, a mechanical tool that allows the camera to rotate independently around its axes, not being integral with the drone's body. As mentioned in the introductory section, UAS, and particularly multicopters, can be very useful to conduct visual inspections and to collect data from a different point of view: they can help human beings to reach elements of the bridge that are difficult to access, enhancing the safety of the inspection. By avoiding the use of by-bridge units, UAS allow not to close the structure to traffic, reducing costs, durations and risks of car accidents due to roadway shrinkage. All these aspects, coupled with the chance to memorize the path of the drone to be used for future activities on the same bridge, make UAS powerful instruments to conduct road infrastructures inspections. Before entering in the field tests, it is important to know what kind of damages bridges are subjected to. First, bridges exhibit different structural defects based on the construction type. One of the most common are reinforced concrete (RC) bridges, that are made of concrete that is reinforced by steel rebar; steel bridges are also widespread. The main causes of bridges degradation are physical, chemical, mechanical and biological processes. Among them, carbonization, chloride pollution, cyclic loads and slow phenomena (such as creep and shrinkage) are the most dangerous and especially affect the structural elements of RC bridges: their most common and dangerous consequences are cracking (that can be divided into spalling and delamination), rebar corrosion and loss of pre-stressing cables stress. It is now clear why periodical inspections and maintenance are essential to prevent and avoid the rupture of structural elements and the collapse of the bridge. As a result, road infrastructures managers establish specific inspection and maintenance plans on their structures; these plans are based on ordinary inspections (that must be done periodically, for example once or twice a year) and furthered inspections (more sporadic or after specific events that may affect the integrity of the bridge). All these inspections need appropriate equipment that allows the examination of structural elements by a so-called "contact distance". All the inspections that do not require the removal of small structural elements can be carried out by using UAS, which enables the detection of all previous listed damages.

CASE STUDIES
In collaboration with Provincia di Piacenza, the administrative authority of a northern Italy town, two tests have been carried out, aimed to conduct a visual inspection and a survey of two bridges near Piacenza. Even though the two bridges are very different, the two tests procedures, instruments and outcomes were very similar and will accurately be described in the next paragraphs.

Lavaiana Bridge inspection
The first field test was carried out on June 13 th , 2019 during a typical spring-like weather that allowed to take advantage of the sunlight for several hours, for a test total duration of about seven hours. The surveyed structure ( Figure 1) is a 140m x 11m x 8m RC bridge that crosses the Lavaiana creek along the SP8 road, made of six spans, five piers and two abutments. In the days prior to the survey, a site inspection was conducted: it is extremely important to evaluate the existing situation before the test takes place. During the inspection, it is possible to establish which instruments are more suitable for the field test and if any natural or artificial barrier to the correct execution of the test exists. In this case, the entire structure was surrounded by a dense vegetation, which represented an obstacle for the drone flight: it was then decided to cut the most invasive trees. As said, the test planning was also set up during the inspection: it was decided to use a DJI Mavic Pro 2 to acquire visual data of the bridge, a Leica Nova Multistation MS60 (MS60) (Fagandini et al., 2017) equipped with a laser scanner to digitally reconstruct the structure with a high level precision and, finally, a Leica Viva GS 14 GNSS antenna to have georeferenced data. GCPs can also be considered as hardware equipment. They consist in a series of enumerated papers, differentiated from each other with a printed mark which facilitates the identification of each portion of the bridge on the acquired visual data. DJI Mavic Pro is a very flexible drone thanks to its small dimensions (about 30cm with open wings) and low weight (less than 1kg), that make it agile but still quite robust. It is equipped with a 20Mpixel Hasselblad camera with a 10bit radiometric resolution that records 60fps 4K video. Due to the remarkable dimensions of the bridge, it was decided to record videos instead of taking pictures manually: this way, the pilot just had to control the UAS position, facilitating the task; a manual flight was preferred to an autonomous one because of the drone path around the structure, which will be discussed below. The MS60 can measure zenith and nadiral angles with a 1'' (0.3mgon) accuracy and it is equipped with a 3D laser scanner with a scanning speed of 1.000 points per second and a millimetric accuracy. During the inspection, three possible station points were identified: by positioning the MS60 in these positions, a complete 3D model of the structure can be obtained by merging each single laser acquisition. Finally, if the outcomes survey have to be included into a global reference system, it is necessary to acquire GPS coordinates of at least three positions around the bridge: to do this, a Leica Viva GS 14 GNSS antenna was used. The first procedures of the test were the GCPs' positioning (for a total of 24, 12 per each side of the structure) and the identification of the multistation positions, whose global coordinates had to be acquired by using the GNSS antenna. Once the GCPs were fixed and clearly visible, their local coordinates could be acquired using the MS60 measurements. At a later stage, GCPs global coordinates were obtained by a rototraslation with a scale factor. At this point, the UAS survey could begin: a total of five flights were manually carried out, acquiring data from a nadiral position, on the east and west side of the bridge and flying around two pillars, the second one and the fourth. These last two flights took place in very narrow spaces, so it was decided to conduct them manually. All the acquired 4K quality videos lasted between 30 seconds and 5 minutes and were characterized by a 29.7fps frequency. With the MS60, alternately positioned in the pre-set positions, a total of seven scans were acquired, all of them with a 5cm x 1cm points network. The first outcome of the field test was the visual inspection of the bridge: typical RC bridges damages were identified, such as spalling phenomena on every pillar, rebar corrosion and moisture spots all over the two abutments. Thanks to the UAS, several structural elements were inspected very quickly and very easily: without having a by-bridge unit or a ladder, bridge bearings conditions were analysed by taking a few seconds video, which can be examined in a later time.
Having performed all the listed actions, the field activity ended, and all the acquired data could be processed in laboratory. To this end, a software equipment was used. Agisoft Metashape Pro (version 1.5.2 build 7838, Windows 64-bit) (www.Agisoft.com, 2019) is a main photogrammetric software allowing several activities starting from the UAS acquired videos. Matlab TM , CloudCompare (version 2.10.2, Windows 64-bit) (Girardeau-Montaut, 2011) and AutoCAD TM were also used: their specific role and characteristics is discussed in this paragraph. The PC used to carry out all the computations described below was an Asus X541UV, with an Intel Core i7-6500U CPU, an NVIDIA GeForce 920MX with 2GB VRAM graphic card and an 8GB RAM. The first laboratory activity was the extraction of the frames from the videos: as a matter of fact, Metashape can only build a 3D model starting from pictures, so it allowed to extract single frames with a frequency established by the user. A 29.7fps video lasting 5 minutes was made up by almost 9.000 frames, which are clearly too many to be given as an input to the software. Therefore, for each uploaded video, a frame step between 5 and 100 was selected, depending on the drone flying velocity and position, corresponding to a frames extraction between 1.2 and 0.4fps. The extracted frames were still too many, so a manual selection of the singles frames was carried out. At the end of this procedure, a total of 302 frames were given as an input to Metashape, together with the GPS coordinates of the GCPs, which were obtained through a rototraslation with a scale factor, starting from their local coordinates. A further step consisted in the "camera calibration" operation. To do this, the selected images were uploaded in Matlab, a numerical computing software, which makes available a proper tool to calibrate the UAS parameters of the camera. At the end of this procedure, an .xml file was generated: it contained the internal orientation parameters (c, ξ0, η0), the focal length (f, mm), the images pixel size (mm) and the geometric distortions of the camera (radial, tangential, affine and skew, for a total of 7 parameters). The last manual activity was the collimation of the GCPs, which consisted in detecting them on every single uploaded image and coupling them with their correspondent GCP coordinates. The more precise this operation, the higher the final model accuracy.
The successive step was the creation of the sparse cloud. It was automatically computed by Metashape through an operation called Photo alignment, and the only user's task was to select the output quality on a five levels scale (from "lowest" to "ultrahigh"): in this case, the sparse cloud was computed selecting a "medium" quality. The output was computed in about 30 minutes and it was made of 66.011 points. Based on the sparse cloud quality and before computing the dense cloud, namely the final 3D model, it was possible to improve the total accuracy by using the "Optimize camera alignment" tool, which allowed to correct possible estimation errors of the parameters, leading to deformations in the final model. Finally, choosing once again a "medium" quality, the dense cloud could be computed through a Metashape command called "Build dense cloud" based on a computation of the collinearity equations system. With a computational time of about five hours, the final output was a 6.042.252 points dense cloud (Figure 2) with an 9mm accuracy; the "medium" quality represents a compromise between a good quality of the model and a reasonable time of computation (Gagliolo et al., 2018;.

Figure 2. Dense cloud of the Lavaiana bridge
It was possible to verify the consistency of the survey by comparing the two clouds of points obtained using the two methods (the photogrammetric and the laser ones). To do this, the M3C2 (Lague et al., 2013) tool of CloudCompare was used to calculate the metric distance between groups of points of each cloud. As inputs, the tool requires a "reference cloud" (the most precise one, generally) and a "compared" one, a "sampling rate" (m) and three additional parameters ("normal", "projection", "max depth"). The laser scanner cloud and the photogrammetric one were respectively selected as "reference" and "compared cloud"; the sampling procedure allowed to reduce the high numerosity of the reference cloud, making the software computation easier. A 0.025m sampling rate was selected: the higher this parameter, the less numerous the output's cloud. A 0.025m sampling rate meant that the new reference cloud could be made by taking one point every 25mm from the original cloud. The subsampled reference cloud was made by about one million points, around 1/6 of the original cloud: they are called "core points" and represent the reference points to calculate the distance between the two models. The "normal" and "projection" parameters generate a sort of cylinder around each core point; their best fit values are generally recommended by the software itself. The last parameter is called "max depth" and establishes the maximum searching distance. As outcomes of this comparison, CloudCompare generated an overall image of the bridge and two histograms: each of them confirmed the good quality of the photogrammetric cloud. In particular, as can be seen in Figure 3 and in Figure 4, along most of the structure, the distance between the two clouds is less than 7mm (light green and yellow colours); the major distances, that are around 3cm and highlighted in blue and red, are less numerous and appear along the bridge's abutments, central pillars and intrados external portions. Considering that the MS60 model precision is greater than the photogrammetric one, these differences can be especially ascribed to three factors: the presence of outliers (or "noisy" points) that were not removed correctly from the sparse cloud; the greater difficulty of reconstruct sharp edges through photogrammetry; or an intrinsic error of the model. As said before, MetaShape's dense cloud is characterized by a 9mm accuracy: this means that each point of the 3D model can differ from the real structure by no more than 2.7cm (with a 99.7% probability). Figure 5. Points' distribution around the average value Therefore, portions of bridge with a distance lower than this can be considered satisfying; on the contrary, portions characterized by a distance greater than 2.6cm from the MS model represent points that do not belong to the structure or structural elements that were not correctly reconstructed. Figure 5 represents the Gaussian statistical distribution of the point, centered around its average value (μ = 0,010m) and with a standard deviation σ = 0.051m. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B2-2020XXIV ISPRS Congress (2020 After these evaluations on the outputs, if the photogrammetric model accuracy is confirmed, the photogrammetric dense cloud can be used as a base of successive operations. As an example, it is possible to use some Metashape proper tools to calculate the extension of spots of humidity in terms of area (m 2 ) and perimeter (m). By exporting the 3D model in AutoCAD TM , this design software was used to draw the bridge's plant, sections and prospects and to metrically measure them.

Lanzone Bridge inspection
The second test was conducted on September 24 th , 2019 during a sunny day, a perfect weather condition for a drone survey. The surveyed structure ( Figure 6) is a 30m x 9.5m x 8m one arch masonry bridge called Lanzone Bridge located in Dignini, a small town in the Vernasca municipality (Piacenza -Italy).

Figure 6. The Lanzone Bridge
Both the equipment and the procedures were the same as the survey for the Lavaiana bridge, except for the fact that this time, because of the small dimension of the bridge, visual data were acquired by taking pictures instead of recording videos. This choice was made during an inspection conducted a couple of months before the survey, during which it was decided to remove some dense vegetation very close to the structure. Besides being a physical obstacle for the flight of the drone, it did not allow to properly reconstruct the covered portions of the bridge. Again, thanks to the size of the bridge, the MS60 computed the 3D model of the bridge by scanning only its east and west side, for a total of two clouds to be merged instead of the seven computed for the Lavaiana bridge. A total of 12 GCPs were positioned on the structure, five on each side and two on the internal walls of the masonry arch. A total of four drone flights were performed, during which 146 pictures were taken. The visual inspection of the bridge highlighted several structural damages, such as deep and extended cracks, humidity spots and vegetation grown between the walls stones. The laboratory test followed the very same procedure of the first one, apart from extraction phase of the frame. All the 146 pictures were given as input to Metashape, which computed a "medium" quality sparse cloud made of 108.904 points with a 20 minutes computational time by using the same PC as before. The dense cloud was computed in about five hours and was made of 5.912.527 points, with a 7mm accuracy (Figure 7). Again, the validity of the outcome was double checked with the laser scanner cloud using the same tool and parameters of the first case study. The outcomes of the analysis were the same as before, including the evaluations made on them. As shown by Figure 8, the distance between the majority of the points of the model is less than 1cm (light green and yellow points). Red and blue points, which represent distances around 3cm, are less than the Lavaiana case and distributed along the upper portion of the structure, characterized by sharp edges. Figure 8. CloudCompare's output Figure 9 represents the statistical distribution of the points, again following a Gaussian curve centered around its average value (μ = 0,000m) and with a standard deviation σ = 0.023m. Figure 9. Points' distribution around the average value As before, once the validity of the photogrammetric model was confirmed, the dense cloud could be used as the basis for the realization of plants, prospects and section by using AutoCAD TM .

CONCLUSIONS
Many road bridges were built right after the World War II, so they are heading to the end of their life cycle. Therefore, to safely perform their function, they must undergo maintenance interventions, which can be substantial. All these interventions are necessarily preceded by inspection and monitoring activities, to determine the structural conditions of the bridge and their evolution over time. As a contribution to the accomplishment of the above activities, UAS is the most suitable instrument: by using this modern technology, visual inspections can be more -0.03 0.00 0.03 10 frequent, well-timed and safer for the workers, who may currently operate in unstable circumstances. Objective and repeatable inspections are further advantages of drones: a bridge that undergoes monitoring activities can be inspected on a quarterly or annual basis by acquiring data from the very same perspective, distance and angles, making them as objective as possible. On this, many proposals are already in place for bridges inspection, such as those from Intel and Flyability (Jordan et al. 2018). By acquiring pictures with photogrammetric characteristics, it is then possible to compute a digital model of the road infrastructure. A major concern about 3D models is represented by the possibility to objectively compare clouds of points that are built during subsequent inspections; furthermore, a digital copy of the inspected structure allows to extract geometrical information. Therefore, for example, it is possible to measure the dimensions of the cracks on the bridge wall and their development over time, monitoring its possible expansion. Each digital model and every information obtained through it allow the realization of a significant database, which in turn can enrich (or build from scratch) a BMS. It is a database that combines inspectors' information, visual and digital data and information extracted from the 3D model, and represents the essential tool for a proper management of the inspection, monitoring and maintenance activities of road infrastructures such as bridges and viaducts. Having such a rich, objective and practical database allows roads managers to efficiently plan every intervention on the bridge, allocating greater resources to the neediest structures, identified among the others thanks to specific indications given by the BMS itself.