DTM GENERATION THROUGH UAV SURVEY WITH A FISHEYE CAMERA ON A VINEYARD

Precision agriculture recommends a sustainable employment of nutrients and water, according to the site-specific crop requirements. In this context, the knowledge of soil characteristics allows to appropriately manage resources. Even the topography can influence the spatial distribution of the water on a field. This work focuses on the production of high-resolution Digital Terrain Model (DTM) in agriculture by photogrammetric processing fisheye images, acquired with very light Unmanned Aerial Vehicle (UAV). Particular attention is given to the data processing procedures and to the assessment of the quality of the results, considering the peculiarity of the acquired images. An experimental test has been carried out on a vineyard located in Monzambano, Northern Italy, through photogrammetric survey with Parrot Bebop 2 UAV. It has been realized at the end of the vegetation season, to investigate the ground without any impediment due to the presence of leaves or branches. In addition, the survey has been used for evaluating the performance of Bebop fisheye camera in viticulture. Different flight strategies have been tested, together with different Ground Control Points (GCPs) and Check Points (CPs) configurations and software packages. The computed DTMs have been compared with a reference model obtained through Kriging interpolation of GNSS-RTK measurements. Residuals on CPs are of the order of 0.06 m, for all the considered scenarios, that for agricultural applications is by far sufficient. The photogrammetric DTMs show a good agreement with the reference one.


INTRODUCTION
Precision agriculture (PA) recommends a sustainable employment of nutrients and water, according to the site-specific crop requirements.In this context, the knowledge of soil characteristics allows to appropriately manage resources, reducing the financial and environmental commitment.The development of proximal surveys of crops and soils, with suitable sensors, vehicles and processing, can give support in defining detailed prescription maps.Obviously, the crop water requirement depends on climate, crop type, growth stage, humidity and soil characteristics.But it is important to remark that also topography can influence soil water condition, because elevation differences control the spatial distribution of water on a field (Schmidt and Persson, 2003).The Topographic Wetness Index (TWI), based on the local slope (Sӧrensen et al., 2006), for example, is a reliable indicator of how topography influences the movement of water and consequently the soil moisture content.The Digital Terrain Model (DTM) describes the topography in a discrete way, with a given resolution and accuracy, and allows deriving TWI maps (Silva et al., 2014).Moreover, in agriculture applications, the crop growth can be monitored from DTM and Digital Surface Model (DSM), by extracting the Canopy Height Model (CHM), computed as filtered difference of surface and terrain heights (Zarco-Tejada et al., 2014).The level of detail of the spatial analysis can be extracted according to the spatial resolution of the DTM.Therefore, the generation of highresolution terrain models has great relevance in agriculture.Different methods can be used in order to obtain an accurate DTM, such as photogrammetry, light detection and ranging (LiDAR), topographic survey, etc.Excluding LiDAR systems, because of their current high costs, employing photogrammetry * Corresponding author in agriculture, as respect to topographic measurements, has the great advantage to be a non-destructive remote sensing technique.In fact, it does not require a direct access to the field, apart for the materialization of few Ground Control Points (GCPs).The discussed work will focus on the production of high resolution DTM in agriculture by photogrammetric processing with a mass market very light Unmanned Aerial Vehicle (UAV).The interest in the use of small consumer UAV for photogrammetric applications is constantly increasing and more and more often these systems are equipped with fisheye lenses.These cameras are characterized by short focal lengths, coupled with a wide field of view, which requires a non-classical projection model (Perfetti, et al., 2017, Kannala andBrandt, 2006).These lenses were firstly embedded in UAV systems mainly for entertainment purposes.However, their use for 3D reconstruction of cultural heritage sites, as well as for precision agriculture applications, has widespread, mainly because of the low-cost, high manoeuvrability and easiness of use of such platforms.The work discussed in this paper is inserted in a project realized in cooperation with the Department of Agricultural and Environmental Sciences -Production, Landscape, Agroenergy (DiSAA), University of Milan, aimed at developing and disseminating the new management practices of PA, in particular to design irrigation systems at variable rate (VR).The main purpose of the project is to optimize and integrate different survey techniques to produce prescription irrigation maps.Information coming from geophysical sensors measuring electrical resistivity (ER), soil analysis to detect the available water content (AWC), topographic survey for DTM and derived TWI, will be merged into the prescription irrigation maps.In Section 1, the study area and the employed UAV system are described, together with the main characteristics of the surveyed campaign.In Section 2, the photogrammetric processing strategy is discussed, as well as the mathematical implementation of the fisheye models embedded in the tested software packages.In the same section the Kriging interpolation method used for the generation of the reference DTM is presented.The obtained results are analysed in Section 3, while conclusions are presented in Section 4.

Study area and UAV survey
A vineyard located in Monzambano (MN), Northern Italy, has been chosen as study area.It covers about one hectare at the Colombara Farm.The study site is shown in Fig. 1.The survey has been conducted using a Parrot Bebop 2, a small lightweight and low-cost quadcopter.It is equipped with 14 Mpx fisheye camera, installed with a fixed inclination of 30°.The use of oblique images acquired with these kind of cameras could be very challenging for commercial photogrammetric software packages.The wide field of view and the tilt angle of the camera can give advantages in agriculture applications because it is possible to investigate the crop not only from nadiral direction, but even from a lateral point of view, to monitor the development of vegetation.The used UAV is characterized by easiness of use and control.It can resist headwinds up to 60 km/h and its flight is quite stable, thanks to the number of sensors installed onboard (namely pressure and ultrasound sensors, GNSS chipset, 3-axis accelerometer, 3-axis magnetometer and 3-axis gyroscope).It can be completely remotely controlled via Wi-Fi connection using a tablet or a smartphone and the flight can be easily planned using the Pix4D Capture App.Because of its low weight, this platform can guarantee a flight autonomy up to 25 minutes.Consumer UAVs, such as the Bebop 2, are equipped with cameras that delivers high quality images, however the use of fisheye lenses for photogrammetric purposes requires a number of specific attentions because the mapping between object and image points is very different from the one that characterize rectilinear images.Because of that, the classical pinhole camera model cannot be used and different optical projections have to be consider (see Perfetti et al. 2017, Barazzetti et al., 2017).Fisheye camera models can be classified considering different type of projections (namely equidistant, equisolid, stereographic and orthographic).However, it is important to point out that apart from the distortion due to the fisheye lenses there are residual distortions that characterize each single lens and are responsible for the discrepancies from the theoretical model.Because of the market widespread of low-cost wide-angles cameras and their use on board UAV platform, fisheye camera models have been added to the most used commercial software packages (e.g.Agisoft PhotoScan Pro and Pix4Dmapper Pro).Moreover, Matlab 2017b introduced a camera calibration procedure dedicated to fisheye lenses.The survey took place on 14 th of December, at the end of the vegetation season, after the pruning of the grapevines.This period of the year was chosen in order to investigate the ground, without any impediment due to the presence of leaves or branches, meaning that for this study the DSM and the DTM can be considered equivalent.In addition, the survey was used as a test for evaluating the performance of the UAV acquisition with Bebop 2 fisheye camera in agriculture, with the aim of carrying out the same procedure during other vegetation seasons for monitoring the growth of the vineyard.A double grid flight was planned with a high longitudinal and transversal overlap, equal to 75% and 70% respectively.The flight height was fixed at 30 m above the ground, ensuring an average Ground Sample Distance (GSD) of about 6 cm.At the end of the survey a total of 215 images were acquired.

Figure 1. The Monzambano vineyard location
A total of 26 black and white (30 cm x 30 cm) and black and yellow panels (50 cm x 50 cm) were placed on the ground and surveyed with a Topcon HiPer SR GNSS receiver (used in Real-Time Kinematic (RTK) mode) in order to obtain the coordinates of the GCPs and Check Points (CPs), which is fundamental to ensure a high geometric accuracy in the generation of the photogrammetric products.Moreover, the coordinates of 15 additional points were acquired with GNSS, in order to generate an independent DTM, through Kriging interpolation, to be considered as reference surface.In Fig. 2, the points used as GCPs represented in red, while all points used for the generation of the reference DTM are shown in yellow.Among these, triangles represent GNSS points employed only for Kriging interpolation, instead squares are points used also as CPs in the photogrammetric process.

Photogrammetric processing
Two software packages widely used in photogrammetry (Sona et al., 2014), Pix4Dmapper Pro and Agisoft Photoscan Pro, have been tested, with the aim to evaluate their performances in case of blocks composed by fisheye images.Moreover, a two-step approach has been realized by pre-processing the images in Matlab 2017b and then performing the bundle block adjustment in Agisoft Photoscan using undistorted rectilinear images.It is worth to notice that these software packages exploit different fisheye mathematical models.Agisoft PhotoScan fisheye model is based on the general form of the equidistant projection.The residual distortions due to lens imperfections are modelled using an extended version of Brown's model (Brown, 1971) combined with the affinity and shear parameters (El-Hakim, 1986).The resulting equations are: where  is the focal length, , ,  are the object coordinate of a generic point,   ,   are the coordinates of the principal point and (∆  , ∆  ) take into account of radial and tangential distortions as well as affinity and shear parameters.Also Pix4Dmapper uses an equidistant model but in this case the incidence angle  is mapped as: The relationship between object and image coordinates is modelled as: where  =  +  2  2 +  3 where , ,  are the object coordinates,  0 ,  1 ,  2 ,  3 are the polynomial coefficients to be estimated during the calibration procedure and  is equal to √ 2 +  2 , with ,  are the ideal projection of the 3D point and   ,   are the coordinates of the principal points.Different image blocks processing strategies have been tested.First of all, the standard double grid block configuration typical of the Bebop system has been evaluated.In order to assess the configurations with a lower number of images, also separate blocks of images acquired along a single direction have been processed, resulting in one block composed by the stripes parallel to the vineyard rows (96 images), and one block composed by strips orthogonal to the rows direction (119 images).Furthermore, three different GCP/CP configurations were considered.In the first one, a quite standard distribution of the GCPs has been used, placing them both inside and along the perimeter of the investigated area.For the second configuration, the internal points have been excluded from the GCPs in order to evaluate the quality of the photogrammetric solution obtainable in those cases where could be difficult or impossible to access the field.Finally, in the third configuration only 4 GCPs, placed at the corners of the field, have been used.The last configuration simulates the operative case of a quick survey, for a rapid production of DTM and orthomosaics, georeferenced on few GCPs.

Pix4Dmapper Pro:
The Bebop system has been developed together with Pix4Dmapper Pro to realize a recommended and standard processing workflow.In fact, this software is able to interpret properly the flight information acquired by the UAV and a good approximation of the embedded camera is stored in its own database.The images have been processed with Pix4Dmapper Pro (version 4.0.25).The tie points search and the estimation of the External Orientation (EO) and Internal Orientation (IO) parameters have been realized using the images at their full resolution, for all the considered scenarios.The dense point clouds have been generated using images with a dimension equal to 1/4 of the original images.Finally, the DTMs have been generated with a ground resolution of 0.40 m.

Agisoft Photoscan Pro:
The images have been processed with Agisoft Photoscan Pro (version 1.4.0),following the standard workflow proposed for fisheye cameras, maintaining their full resolution (correspondent to the high alignment quality of the software).A previously estimated set of Internal Orientation (IO) parameters has been used as initial camera calibration.This has been necessary because of the non-coherent pixel size value read by the software from the EXIF file, with respect to the nominal focal length.These approximate parameters have been then refined using PS 'optimize stage'.The dense clouds have been generated downgrading the images with a factor equal to 4 (i.e. using the high quality of Agisoft Photoscan Pro), guaranteeing the same level of resolution used by Pix4Dmapper.Also in this case, the DTMs have been generated with a ground resolution of 0.40 m, for all the considered scenarios.

Matlab 2017b + Agisoft Photoscan Pro:
A two-step procedure has been tested for evaluating the effect of preprocessing the fisheye images and transforming them into undistorted rectilinear images.A new set of functions that allows to calibrate fisheye images and correct them from lens distortions have been embedded in Matlab 2017b.After a dedicated calibration procedure (realized using the Matlab chessboard pattern), undistorted rectilinear images have been generated, together with the parameters of a virtual perspective camera that produce those images.These images have been processed in Agisoft Photoscan Pro, maintaining the same level of resolution previously illustrated.Because any UAV system suffers impacts during take-off and landing, a self-calibration has been performed in Agisoft Photoscan Pro by using the 'optimize stage'.Even if the area covered by the rectilinear images is smaller with respect to the fisheye ones, the image overlapping stays still quite high (>9), which does not affect the completeness of the generated 3D models.The DTMs have been generated with a ground resolution of 0.40 m.

Bundle block adjustment results:
The quality of the photogrammetric solution has been evaluated considering the residuals of the CPs, for all the scenarios.The results are shown in Table 1.They are all in line with the requested tolerance, indeed the highest 3D residuals are of the order of 0.07 m in the worst case.From here on, Pix4Dmapper Pro is defined as P4, Agisoft Photoscan Pro as PS and the processing with Matlab 2017b and Agisosoft Photoscan Pro is identified as MPS.
Considering the different flight configuration, it is quite evident that the better results have been obtained for the double grid configuration, with 3D residuals of the order of 0.03 m for all the evaluated software packages, and horizontal residuals around 0.015 m.This is quite reasonable because such flight path ensures the strongest acquisition geometry, which reflects in a better intersection of the homologous rays.Moreover, it is quite evident that the use of internal GCPs has no meaningful impact on the final accuracy.As expected, the use of only four corner GCPs slightly reduces accuracy in all double grid cases.
For both MPS and P4 the worst results have been obtained for the block composed only by the images acquired along the vineyard rows.Instead, for PS there is slight worsening for the block composed by the images acquired orthogonal to the vineyard rows, especially along N and h coordinates.

Kriging interpolation
The reference DTM has been obtained by interpolating 15 points, whose coordinates have been measured with GNSS-RTK.This low number of observations has been considered sufficient for the estimation of the DTM because the investigated area is quite flat, without significant topography variation.The DTM has been obtained by interpolation using the Kriging method, in the version implemented in the Geostatistical Wizard of ArcMap (version 10.5).The use of a geostatistical method has been preferred to the use of a deterministic methods (such as Inverse Weighted Distance, spline etc.) because it provides also a standard map error, that shows the uncertainty related to the predicted values.(Burrough and McDonnel, 1998).The reference DTM has been created using the ordinary Kriging, which assumes that there is an unknown constant mean value and that the phenomenon is continuous in the space.These assumptions can be considered correct in case of a field characterized by a flat topography, without a pronounced trend.
Of course, the model created with an interpolation procedure tends to be flatter with respect to the DTMs generated from photogrammetry, because it does not capture local spatial variability (e.g.minor ground furrows).The resulting interpolated model is shown in Fig. 3, while the standard error map is reported in Fig. 4. The hypothesis of almost flat field is respected, as the maximum variation in 150 m (along the diagonal of the field) is less than 1.5 m.Moreover, errors are lower than 0.03 m for most of the area; as expected the highest error values are located along the edges.

RESULTS
As a first general remark, it can be stated that all photogrammetric process strategies gave satisfactory results, as even the worst accuracies are around 0.06 m, that for agricultural applications is by far sufficient.All the DTMs, obtained with different image block configurations, number of GPCs and software packages, have been compared with the reference one, generated by Kriging interpolation.This DTM is considered as reference because point interpolation describes the main behavior of height field, neglecting the high frequency details due to lines and interlines of the vineyard.For the sake of brevity, only few comparisons are here analyzed and deeply discussed, choosing the better cases.Crossing the information that double grid blocks have produced the better accuracies, and at the same time the use of GCPs in the middle of the field have not improved results, we can consider as 'best result', for each processing type (MPS, PS or P4) the block formed with double grid and 9 GCPs on the border.In Fig. 5, the differences between the reference DTM and the photogrammetric models obtained with different software packages are shown.It is quite evident, that the computed differences underline the presence of the furrows for all the cases.This is because the photogrammetric models represent the terrain with a higher level of detail, while the interpolated model is smoother and correctly represents only low frequency phenomena.Moreover, all the computed maps are characterized by a common spatial pattern, which reflects the spatial distribution that can be observed in the error prediction map, generated during the Kriging interpolation (see Fig. 4).The highest differences between the reference DTM and the photogrammetric ones are of the order of 0.2 m (absolute values).Table 2 reports mean and standard deviation values for the computed differences.The values show a good agreement among the models.Considering that the standard deviations assume the same values for all the cases, P4 model is the most similar to the reference one.This can be due to the better performance of this software, with processing options specifically designed for Parrot Bebop fisheye imagery.Even if, a certain level of smoothing due to interpolation is inevitably introduced by each software packages during DTM generation, it has been noticed that P4 point clouds were in general smoother than point clouds generated with PS and MPS.This is probably due to the fact that tie-points search is performed by using a sky-mask, which avoid creating erroneous matchings that increase the roughness of the computed point cloud.The smooth behaviour of DTM produced by Kriging interpolation is evident even in the section, shown in Fig. 6.The photogrammetric models well describe the terrain profile with the high frequency component given by the presence of furrows and lines.In addition, the profile confirms that the DTM generated by P4 software is the closest to the reference one.

CONCLUSIONS
This work investigates the use of Parrot Bebop 2 fisheye camera for generating very high resolution DTM to be employed for precision agriculture purposes, as well as prescription irrigation maps.All the tested strategies have given good results and this study has been useful for defining best practices for surveying and processing optimization.It is possible to obtain accurate DTMs without any GCPs placed inside the field, as long as GCPs are in sufficient number for the photogrammetric process and well distributed all around the surveyed area.Image pre-processing, aimed at removing fisheye distortion to produce standard frame images, is not necessary and not advantageous in terms of computational time.Regarding Agisoft Photoscan Pro, recent releases of the software can deal with fisheye cameras, but high quality processing is not worthy in terms of requested time with respect to obtained accuracies.Double grid flight configuration should be preferred to single direction flight.In order to reduce the acquisition and processing time, it should be considered to reduce the number of strips and images.This implies a reduction in transversal and longitudinal overlaps (Ajayi et al., 2017).The GSD high variability through a single image and the real overlapping areas must be carefully taken into account.
To check the accuracies obtainable with this strategy, further processing tests should be performed, by forming image blocks selecting one every two images, and one every two strips.

Figure 2 .
Figure 2. Distribution of points used for the generation of DTM by interpolation and for the photogrammetric process

Figure 3 .
Figure 3.The DTM created by Kriging interpolation Table 2. Means and standard deviations of the DTM differences a) DTM generated with P4 software and extracted section (blue line).b) Terrain profiles extracted from the different DTMs.
Scaramuzza et al. (2006)are the coefficients of a polynomial function, , , ,  are the coefficients that allows to map the undistort image coordinates into the distorted ones (  ,   ).The diagonal element of this matrix can be related with the focal length.The fisheye model embedded in Matlab 2017b release is based on the general model for calibrating omnidirectional cameras discussed inScaramuzza et al. (2006). Themapping function is represented by a Taylor series expansion, whose coefficients are estimated via 4-step least square adjustment.The resultant model for fisheye lenses is described by:  2  2 +  3  3 +  4  4 ) +

Table 1 .
Residuals on the CPs after bundle block adjustment