THE USE OF THE UAV IMAGES FOR THE BUILDING 3D MODEL GENERATION

: The growing interest in recent years in Unmanned Aerial Vehicles (UAVs) by the scientific community, software developers, and geomatics professionals, has led these systems to be used more and more widely, in different fields of engineering and architecture. This is thanks, above all, to their flexibility of use and low cost compared to traditional photogrammetric flights using expensive metric digital cameras or LiDAR sensors. In recent years, UAVs have also been used in the field of monitoring and inspection of public or private buildings that are remarkable in terms of size and architecture. This is mainly due to the focus a sustainability and resource efficiency in the building and infrastructure sector, which aims to extend their lifetimes. Through the use of remote checking using UAVs, the monitoring and inspection of buildings can be brought to a new level of quality and saving. This paper focuses on the processing and study of 3D models obtained from images captured by an UAV. In particular, the authors wanted to study the accuracy gains achieved in the building 3D model obtained with both nadir and oblique UAV flights. The images from the flights were processed using Structure-for Motion-based approach for point cloud generation using dense image-matching algorithms implemented in an open source software. We used the open source software VisualSfM, developed by Chanchang Wu in collaboration with the University of Washington and Google. The dense matching plug-in integrated in its interface, PMVS/CMVS, made by Yasutaka Furukawa, was employed to generate the dense cloud. The achieved results were compare with those gained by Photoscan software by Agisoft and with 3D model from the Terrestrial Laser Scanner (TLS) survey.


INTRODUCTION
The growing interest in recent years in Unmanned Aerial Vehicles (UAVs) by the scientific community, software developers, and geomatics professionals, has led these systems to be used more and more widely, in different fields of engineering and architecture.This is thanks, above all, to their flexibility of use and low cost compared to traditional geomatic survey as photogrammetric flights with expensive metric digital cameras or LiDAR sensors (Ezequiel et al., 2014).Other advantages are the reduced image acquisition time and the ease of images processing thanks to the presence on the market or web of low-cost or free/open source softwares.UAV systems were used first for military purposes and later for civilian ones, such as agricultural and forest management (precision farming, assessment of woodlots, fire surveillance, etc.) (Remondino et al., 2011;Aicardi et al., 2016), archaeology and cultural heritage surveying (3D documentation and mapping) (Berni et al., 2009;Chiabrando et al.;2011), for 3D reconstruction (Chiabrando et al., 2012.) and others.Specifically, in recent years, UAVs have also been used in the field of monitoring and inspection of public or private buildings that are remarkable in terms of size and architecture.This is mainly due to the focus a sustainability and resource efficiency in the building and infrastructure sector, which aims to extend their lifetimes.This type of monitoring and inspection has always required sophisticated and expensive methods to quickly and safely identify possible damage that could jeopardize the stability and safety of buildings and people.Through the use of remote checking using UAVs, the monitoring and inspection of buildings can be brought to a new level of quality and saving.The buildings 3D model, 3D, particularly those of historicalarchitectural importance, become of strategic importance also for Historical Building Information Modeling (HBIM) or for the creation of 3D City Models or GIS 3D (Deidda et al., 2013, Deidda et al., 2015) of our cities, allowing in this way a better knowledge and management of the building (Vacca et al., 2018).Finalized to the full achievement of these last objectives, and in particular for a complete geometric reconstruction of buildings with high accuracy, from several years the nadiral and oblique UAV flights are used (Vacca et al.,2017).The use of oblique images allows the reconstruction of 3D models of buildings including façades.Furthermore, this technique permits the measurement and survey of the higher parts of the objects, which are not easily achievable by the common survey techniques, such as terrestrial laser scanning (TLS) or terrestrial photogrammetry (Lingua et al., 2017).This paper focuses on the processing and study of 3D models obtained from images captured by UAV.In particular, the authors wanted to study the accuracy gains achieved in the surveying and the measurement of the dimensions of the buildings in the 3D models or their health state obtained with nadir/oblique UAV flights.These types of flights are particularly suitable for the 3D modelling of cities or urban agglomerations, where it is important to achieve a complete building reconstruction, including façades and footprints of buildings.For this purpose, some UAV surveys with both nadir and oblique axes were performed on the buildings.The images from the flights were processed using Structure-for Motion-based approach for point cloud generation using dense image-matching algorithms implemented in an open source software.In particular we used the open source software VisualSfM, developed by Chanchang Wu in collaboration with the University of Washington and Google, was used for the images alignment by means of the SfM technique (Wu, 2011;Wu, 2007).The dense matching plug-in integrated in its interface, PMVS/CMVS, made by Yasutaka Furukawa, was employed to generate the dense cloud (Furukawa, 2010).The achieved results were compare with those gained by Photoscan software by Agisoft and with 3D model from the Terrestrial Laser Scanner (TLS) survey.

Instruments
The purpose of this research is to study the accuracy of 3D models obtained from nadiral and oblique imagery acquired by UAV systems and processed with the software free VisualSfM.In order to achieve these goals, we performed an UAV flight with nadir and oblique images on a single building.
The UAV system used is the FlyNovex UAV by FlyTop S.p.A.
(Figure 1).It's a hexacopter weighing 6 kg with a 5 km range and 20 minutes of autonomy.The system contains an IMU (inertial measurement unit), a GPS receiver and a camera mounted on the gimbal.The UAV has a wireless connection with a computer working as the ground control station.The camera is a Sony Alfa 6000 model with a resolution of 6000 pixel x 4000 pixel, a focal length of 16 mm, sensor size of 23.5 mm x 15.6 mm and a pixel size of 3.92 micron.The camera's calibration parameters are reported in Table 1.
The images were processed using VisualSfM, a GUI (Graphical User Interface) application for 3D reconstruction, devoloped by Chanchang Wu in collaboration with the University of Washington and Google, was used for the images alignment by means of the SfM technique.
The SfM is a low-cost photogrammetric method for highresolution topographic reconstructions.The SfM operates under the same basic tenets of the stereoscopic photogrammetry, namely that the 3D structure can be resolved from a series of overlapping images.However, they fundamentally differ because in SfM, the geometry of the scene, camera positions, and orientation is solved automatically without points known.The points are solved simultaneously using a highly redundant, iterative bundle adjustment procedure, based on a database of features automatically extracted from a set of multiple images with a high degree of overlap.The approach is most suited to sets of images with a high degree of overlap that capture the full threedimensional structure of the scene viewed from a wide array of positions, or as the name suggests, images derived from a moving sensor (Szeliski, 2010;Westoby, 2012).
The building 3D model obtained from the UAV flight was validated by comparison with the 3D model from a terrestrial Laser Scanner (TLS) survey.The instrument used was a Faro Focus 3D.This is a compact scanner characterized by an operative range that varies between 0.6 m and 120 m, with a linear distance error of ±2 mm for scanner-object distances comprised between 10 m and 25 m.It has a vertical visual field of 305° and a horizontal one of 360°.The vertical and horizontal resolution is 0.009°.The processing of the scans was done using the JRC Reconstructor Software v. 3.1.0(335) by Gexcel Ltd. (Spin Off of the University of Brescia, Bergamo, Italy).Reconstructor is a software that enables all the processing operations of the point cloud before the 3D model of the structure or building scanned is obtained.
The comparison was done both comparing the measurements of distances taken on the point clouds obtained from UAV 3D model and TLS surveys, and comparing the point clouds themselves using the CloudCompare software (Open Source Software).Specifically, the comparison was done by calculating the minimal distance between every point of the models using the nearest neighbor algorithm.Furthermore, the software allows the calculation of statistical values, such as the minimal distance, maximal distance, average distance, and standard deviation.
The images were processed also with a similar well-known proprietary commercial software, Agisoft Photoscan, (www.agisoft.com)which reaches very high performances and accuracies, as a number of studies proves (Remondino, 2014).
The VisualSfM and Photoscan work in the same way and they are following these steps: images import, image alignment, generation of the sparse cloud, georeference and dense image matching.

The nadir/oblique UAV flight
For this study we planned and executed a flight over a single building and we validated the results with a TLS survey done directly by us.The chosen building was an abandoned construction with a regular shape, about 12 m tall.The flight was characterized by images acquired with nadiral axis and with an axis inclined by 45° with respect to the nadiral direction.Nowadays, considering the case of oblique images, the question on the modalities through which flight configuration allows covering the entire building or object is still open (Lingua, 2017).
In the recent years, several configurations have been tested and evaluated (Rossi et al., 2017): the Maltese Cross with a nadir camera and four oblique ones (45°) pointing in the cardinal directions, and the Fan configuration that increases the swath width along the track to cover more area.In our case, the figure 2 shows our oblique flight that was done with a 45° inclination of the optical axis.the Depth Filtering to "Moderate".Georeferencing point clouds were made using four GCPs on the terrain and five on the façade (Figure 3), surveyed in the ETRF2000 datum (Barbarella, 2014).The r.m.s of the georeferenced image processing have been attested to a few centimetres.
It is important to point out that the nadir flight was realized in fully automatic mode, while the oblique flight was in manual mode.The pilot, in this case, with the help of the remote viewing camera, made the shots attempting to cover the required 80% overlap.From the pre-processing of the images, however, it has been verified that the overlap of 80% was guaranteed.The TLS survey was done using a Faro Focus 3D laser scanner, with four exterior scans (figure 4) with a resolution of one point per 7.67 mm at 10 m (Vacca et al., 2016;Deidda et al., 2012).As can be seen from table 3, the number of points processed by Visual SfM is comparable with those of the point cloud processed by Photoscan, while both are much lower than that of the TLS survey.
The point clouds obtained from the processing were validated in two different ways.The first one consists in taking measurements L1, L2 and H (figure 5) on the building through the different point clouds; the other one, by calculating the minimal distance between every point of the 3D model obtained from each flight and the 3D model produced by the TLS survey.This calculation used the Nearest Neighbor algorithm of the Cloud Compare program.The table 4 reports the comparisons between each quantity measured on the 3D model.
The analysis of table 4 shows that the measurements made on the point cloud generated by VisualSfM differ from those coming from the TLS of a few tens of centimeters, while those obtained from Photoscan are slightly worse.The second validation comported a direct comparison between the point clouds from the UAV flight and from TLS survey, through the calculation of the minimal distance between every point.For practical reasons, this comparison did not interest the whole building but only the northern façade (figure 6), which did not contain any eaves or obstructions, thus ensuring that the statistical results depend only on the processing and not on the geometry or the contour conditions of the examined element.Table 5 reports

CONCLUSION
The paper describes a UAV methodology and instrumentation for 3D surveying of buildings through the acquisition of nadiral and oblique images.In particular we wanted to test the accuracy and functionality of an open source software and compare it to those already extensively tested by Photoscan, as well as to the 3D model coming from a TLS survey.The use of VisualSfM was easy and immediate, with faster processing times especially compared to those of Photoscan.It would be useful, however, to be able to customize, via the input of parameters, the management of the elaborations from the creation of the sparse cloud to the dense one, through the use of PMVS/CMVS.Another problem encountered was the georeferencing, which was a long and not very automated process.Overall, however, we can say that the open source software VisualSfM is really useful in all those contexts where one needs a 3D reconstruction of buildings quickly and accurately, aimed at the geometric knowledge of the building, at the analysis of his health or as the basis for a possible conservative restoration project.The research will continue by applying the same methodology to buildings of historical-architectural importance to verify the correspondence of the processing of VisualSfM software to architectural details whose modeling could be more complex.Furthermore, the 3D models generated can be a good basis for the creation of 3D City Models or 3D GIS.

Figure 2 :
Figure 2: Oblique flight The TLS point clouds were georeferenced on the same GCPs used for point clouds obtained by UAV flight, with r.m.s. of 3.5 cm.The final 3D model is composed by 5.203.483points and the r.m.s. of the registration is 2 mm.

Figure 5 :
Figure 5: The measurements taken on the building

Flight 5 :
the number of points of the portion of point cloud representing the north façade for each configuration.Number of points North facade Also from table 5 it is confirmed that the result on dot density is better in processing with Photoscan respect to VisualSfM.The calculation of the minimal distance between every point of the point clouds obtained from the different software packages and the single cloud obtained from the TLS survey was done using the CloudCompare software, which compares point clouds representing the same object but acquired in different times and/or with different instruments.CloudCompare also calculates the following statistics: minimal distance, maximal distance, average distance and standard deviation.

Figure 10 :
Figure 10: Discrepancy histogram between the TLS cloud and UAV Photoscan processing.

Figure 11 :
Figure 11: Discrepancy map between the TLS cloud and UAV VisualSfM processing.

Figure 12 :
Figure 12: Discrepancy histogram between the TLS cloud and UAV VisualSfM processing.

Figure 13 :
Figure 13: Discrepancy map between the UAV Photoscan processing and UAV VisualSfM processing.

Table 1 :
Calibration parameters of the camera Table2shows the parameters of the flight plan for the nadir and oblique flights.The images from the UAV flight were processed using VirtualSfM and Photoscan.The dense point cloud was calculated with Photoscan setting the Quality parameter to "Medium" and

Table 3
reports the number of points recorded in the different point clouds representing only the building.

Table 3 :
Number of points in the 3D model

Table 6 :
Table 6 reports the results of the comparisons between the point clouds obtained with TLS and the point clouds from VisualSfM and Photoscan processed.Statistical values of the comparisons between the TLS data and the point clouds Table 6 shows that statistics on point clouds from VisualSfM and Photoscan are comparable.Only the VisualSfM standard deviation is double that of Photoscan but always below 10 cm.Figures 9, 10, 11 and 12 show the discrepancies maps and histogram between the TLS point clouds and the ones obtained from the UAV surveys.

Table 4 :
Comparison between the measurements

Table 7
shows comparisons between VisualSfM and Photoscan processing.

Table 7 :
Statistical values of the comparisons between the Photoscan data and the VisualSfM data