CALIBRATION OF LOW COST RGB AND NIR UAV CAMERAS

Non-metric digital cameras are being widely used for photogrammetric studies. The increase in resolution and quality of images obtained by non-metric cameras, allows to use it in low-cost UAV and terrestrial photogrammetry. Imagery acquired with non-metric cameras can be used in 3D modeling of objects or landscapes, reconstructing of historical sites, generating digital elevation models (DTM), orthophotos, or in the assessment of accidents. Non-metric digital camcorders are characterized by instability and ignorance of the interior orientation parameters. Therefore, the use of these devices requires prior calibration. Calibration research was conducted using non-metric camera, different calibration tests and various software. The first part of the paper contains a brief theoretical introduction including the basic definitions, like the construction of non-metric cameras or description of different optical distortions. The second part of the paper contains cameras calibration process, details of the calibration methods and models that have been used. Sony Nex 5 camera calibration has been done using software: Image Master Calib, Matlab Camera Calibrator application and Agisoft Lens. For the study 2D test fields has been used. As a part of the research a comparative analysis of the results have been done. * Corresponding author


INTRODUCTION
Currently, the most common and most frequently used in the Unnmaned Aerial Systems (UAS) are compact digital cameras (non-metric cameras).Metric cameras are able to guarantee the stability of the internal orientation elements.Typically, these cameras do not have the possibility of zoom lens.In contrast, non-metric cameras are characterized by instability and ignorance of the elements of internal orientation.Cameras used for photogrammetric purposes are usually compact cameras or SLR (Single-Lens Reflex Camera).Compact cameras are cameras with a small size and simple design, equipped with a fixed lens (fixed focus lens or zoom lens).The lens retracts into the camera body.Focus or exposure time is carried out automatically.The viewfinder is placed above the lens, causing the phenomenon of parallax .These cameras have a much smaller matrices than an SLR or hybrids .compact Cameras having a fixed lens have smaller geometric resolution than cameras having a zoom lens .While the design of SLR cameras now prevails among the top and middle class.There is only one lens, which is used at the same time to observe the frame and exposures.The advantage of this type of camera is to give the frame in the form in which they will be exposed .Another advantage is the total lack of parallax.Lenses with large focal lengths have built-in stabilizers, which allow you to make sharp images without a tripod.The SLR lenses are used so-called zooms and fixed lens.
Digital camera consists of elements such as: body, bayonet lens, image sensor, LCD display, memory card.The matrix is the arrangement of light sensitive elements.The task of each element is registration and ability to read the electrical signal that is proportional to the amount of incident light to the matrices .The digital cameras are most frequently used color filters so that each of the sensor reads the intensity of the predetermined width of the light spectrum.Matrices are divided into two types of structures of their construction: CCD (ang.Charge Coupled Device) and CMOS (ang.Complementary Metal-Oxide-Semiconductor).They differ in production technology.The CCD has one converter (analogdigital) that processes information from each pixel into a digital signal.Data is read pixel by pixel.However, in the CMOS sensor, each pixel has its own transmitter , so that the matrix is faster (faster recording images and the ability to perform more shots).The array has on its surface a greater amount of circuitry that can block light entering the inside of the pixel.More and more the amateur digital cameras are used for photogrammetric purposes.Camera selection (metric or nonmetric) depends on the desired accuracy.For topographical applications, architectural inventory, or in cases where it is required to obtain measurements with very high accuracy, photogrammetric cameras are used (metric).In contrast, digital compact cameras are used in cases where they may get lower measurement accuracy for example when measured special objects.The use of digital cameras for photogrammetric purposes requires calibration.This is due to the fact that the non-metric cameras elements of internal orientation are unknown .In connection with the increase in resolution and quality of images obtained with non-metric cameras they are used in aerial and terrestrial photogrammetry.Non-metric images can be used for modeling 3D objects, reconstruction of monuments, to generate digital elevation models (DEM), orthophotos, or for the assessment of accidents.

RELATED WORKS
Camera calibration and the evaluation of high-quality interior orientation parameters is one main topic in research and development of photogrammetry since decades.(Hastedt and Luhmann, 2015).Architectural documentation is carried out mostly with non-metric cameras or video-cameras.Such experiences have been shown in (Delis et al., 2013).Unknown internal geometry is a main problem in this context, particularly for wide-angle lenses with their considerable amount of distortion.Some aspects of calibration of non-metric cameras were presented in (Karras and Mavrommati, 2001).Other publications that threats about the calibration of non-metric camera calibration methods are (Fraser and Remondino, 2006), (Honkavaara et al., 2006), (Harley et al., 2006), (Kedzierski and Fryskowska, 2008) or (Wang et al., 2009).The work that presents results of surveying two objects of interest for the conservation of the architectural and cultural patrimony of the Old Havana with the final verification of results is presented in (Rodríguez et al., 2008).Currently, non-metric cameras are mounted in the Unmanned Aerial Systems.In this case, also interior orientation information is needed.Such experiences describe (Pérez et al. 2011), (Douterloigne et al., 2009), (Kedzierski, M., Wierzbicki, 2015), (Dabrowski et al., 2014)

Lens distortion
In non-metric cameras, there are two types of distortion: radial and tangential (tangent).The device, which is ideal lens angle at which the beam enters main radius (it is one that after passing through the lens is parallel to the incident beam) is equal to the angle at which the beam exits.The distortion is related to the structure of the lens -the distribution and the value is already known in the design stage of the lens, so the radial distortion is defined as the theoretical distortion.The value of distortion ∆r depends on field angle (the angle between the axis of the lens and the main radius) and the radial distance r.
The curve of the radial distortion can be represented using a polynomial of odd powers of the radial distance. (2) where: K 1 , K 2 , K 3polynomial coefficients of distortion; rradial distance. (3) where: x, yimage coordinates of the point; x 0 , y 0image coordinates of the principal point.
To describe the radial distortion curve we need to know only two components of polynomial distortion.For analytical description it is required to determine a radial distortion coefficients: K1, K2 and K3. (4) where: ∆x r , ∆y rthe effect of radial distortion for image coordinates x, y; y ̅ , x ̅image coordinates relative to the principal point.
Radial distortion in the real lens is not strictly symmetrical (different from the theoretical distortion).For different radii of radial distortion do not coincide with each other.

METHODS OF NON-METRIC CAMERAS CALIBRATION
For non-metric cameras the elements of internal orientation are instable and unknown.Any type of conventional camera, which does not require any additional parameters (fiducial marks) to define the internal orientation are referred to as non-metric cameras.
The main problem of non-metric images is that they do not guarantee us stability elements of the interior orientation (their repeatability).Measurement of fiducial marks associated with the principal point is impossible, because they do not occur in images taken by the non-metric digital camera.The role of fiducial marks in the images of the non-metric cameras perform extreme corners of the image pixels.
The process of determination of internal orientation parameters is calibration.During calibration we determine: focal length, principal point, radial distortion, tangential distortion, and other.(Harley et al., 2005) There are several methods of camera calibration: laboratory, field, self-calibration.Laboratory calibration requires the calibrator and usually is applied for the metric cameras.The file calibration method requires known reference filed.The method bases on the images of this test field.The test field consists of a large number of points that evenly cover the entire frame.Signalized points are imaged in the picture.There are flat test fields -2D and spatial -3D.Self-calibration is the process of camera calibration performed during the measurement process.This is due to the increased number of images taken, and the necessity of the use of specialized software.The greater the number of images, and thus the greater the number of measured homologous points, the accuracy is higher.

Cameras used
Two non-metric cameras were used: SONY NEX 5 with two types of matrices: RGB (SONY NEX 5R) and NIR (SONY NEX 5T).Both cameras have the same structure like Sony NEX5 compact camera, with the APS-C Exmor CMOS matrix (format -3:2) with the maximum resolution of 4912 x 3264.The size of the matrix is 23,5 x 15,6 mm (Figure 1). Figure 1.Bayer filter (Nikon, 2016) Two green pixels falls one pixel red and blue.To create a multicoloured image of this combination of points, interpolation Bayer have to be used.It bases on the estimation of the light intensity measured by the neighbouring pixels, missing information for a particular colour.The camera therefore operates in the visible range from 400 to 720nm .However, in the case of cameras Sony NEX 5T cell sensitive to the colour blue is replaced with a cell sensitive to NIR range, so the camera takes the image in the range from 500 to 1200 nm.In addition, in the camera additional filter for transmitting infrared is mounted.In this case, the selected filter is HOYA R72 with a diameter of 55mm (Figure 3).It is a filter which absorbs all visible light to a length of 720nm, and transmits infrared radiation.

Matlab R2015a -Camera
Calibrator software was created in order to determine the parameters of the interior orientation of digital cameras (non-metric).It enables to determine the eight calibration parameters: focal length f x , f y , principal point coordinates x 0 ,y 0 , radial distortion coefficients: K 1 , K 2 and tangential distortion coefficients: P 1 , P 2 .The test field reminds chessboard (Figure 5).The test consists of a squares: white and black, which are arranged alternately.
One page should contain an even number of fields, and the other odd .In this way, the pattern comprises two black corners along one side and white two corners on the opposite side.The calibration sheet cannot take the shape of a square, as arranged along a long side axis X, but along the short axis Y.This allows to specify the orientation of the pattern.However Agisoft software also has the ability to perform selfcalibration for non-metric images in two other modules (Agisoft PhotoScan and Stereoscan).In this software we can determine the same calibration parameters like in the Camera Calibrator application.As in the case of the Camera Calibrator for calibration chessboard is used.

RESULTS
10 series of images have been taken with Sony NEX cameras, which resulted in 100 images per test field.The calibration images should be taken under similar conditions to the field samples for most accurate results.(Pérez et al., 2011) Calibration have been done in three mentioned software.
The results of two non-metric cameras calibrations in Image Master (interior orientation parameters) are presented in Tables 1 and 2. Constant f for both cameras has been determined with very high accuracy, as presented by the low standard deviation values.The calculated value is different from the theoretical manufacturer's focal length of 15 mm by 0.04 mm .In the case of the other theoretical parameters manufacturer did not give their value, and therefore they can not be compared.

Parameter
When comparing the values obtained for the two non-metric cameras the same manufacturer can be stated that the results do not differ much from each other.Focal length of both cameras have reached an almost identical value that is 14.96 mm.In the case of principal point coordinates two main values differ from each oher by the tenth millimetre.It should be noted that the standard deviation of these values are small -amount to hundredths of millimetre.Also very low standard deviations for the second factor of a polynomial has been reached.
In comparison, in Tables 3 and 4  Focal length values for Sony Nex 5R camera (theoretical and calculated) varies by 0.1 mm.It is a small difference, most likely due to application errors.
However, in the case of calibration parameters for camera Sony NEX 5T, the results are very different from the other, eg .focal length is different from that specified by the manufacturer to 0.5 mm for Matlab and 0.4 mm for Agisoft.

CONSLUSIONS
The paper presents the process, the results and the accuracy of two calibrations methods.After the software processing, the camera calibration parameter values were obtained.the analyzes show that the most accurate deployed software program was Image Master .It achieved the highest accuracy of all programs for the majority of calibration parameters .for the calibration of digital cameras have been used for nonmetric photogrammetry typically best results (stability and accuracy has been obtained I Image Master Calib.Also processing time and complexity calculations is the best using this sotfware.

Figure 4 .
Figure 4. Test filed for Image Master Calib

Figure 5 .
Figure 5. Test field for Camera Calibrator

Table 4 .
The results of Sony NEX 5T calibration(Matlab and  Agisoft) calibration results for two cameras in two other software packages (Matlab and Agisoft) have been presented.