RAPID RADIOMETRIC CALIBRATION OF MULTIPLE CAMERA ARRAY USING IN- SITU DATA FOR UAV MULTISPECTRAL PHOTOGRAMMETRY

This study compares two possible radiometric calibration approaches of Tetracam μMCA Snap multispectral camera using the Dark offset subtraction and Look-Up Table (LUT) methods. A laboratory-based calibration using correction images taken under the controlled conditions was compared with a rapid in-situ based calibration featuring correction images taken during the field campaign. The hypothesis was that the accuracy of in-situ calibration could be comparable with the laboratory calibration, and thus could replace it and simplify the radiometric calibration process. The accuracy of calibration approaches was assessed by comparison of three validation targets reflectance values extracted from corrected UAV images based on laboratory and in-situ calibration with a reference spectroscopy measurement. The results of the field experiment showed that both calibration approaches led to significant accuracy improvement compared to raw data. The vignetting correction using resulted in a significant reduction of the Coefficient of variation by half in all bands and overall equalizing the DNs on the selected diagonal profile. The NRMSEs after processing all corrections ranged from 0.24 to 3.40%. Although the statistical testing revealed slightly better agreement of laboratory calibrated reflectance with reference data, the accuracy of in-situ calibration is sufficient, because the accuracy improvement quantified by the NRMSE is 2 – 10 times better using both calibration approaches compared to raw data than the NRMSE differences between them. These findings make the proposed in-situ approach usable for various environmental studies featuring UAV multispectral photogrammetry.


INTRODUCTION
Unmanned Aerial Systems (UAS) featuring multispectral sensors are popular remote sensing instruments for rapid mapping and monitoring of dynamic phenomena in geosciences, because they have lower operational costs, higher spatial resolution and flexible temporal resolution compared to conventional remote sensing (Manfreda et al., 2018;Minařík and Langhammer, 2016;Zhang and Kovacs, 2012). Moreover, UAS can be operated on demand in often varying conditions, which increases the importance of radiometric calibration in the image processing (Aasen et al., 2018). The raw spectral data are altered by a mixture of effects that include the current surface conditions, atmospheric effects, topographic effects, and sensor characteristics (Smith and Milton, 1999). These alterations of the original digital numbers (DN) limit the quality and validity of the raw data for further photogrammetric processing and time-series analysis in environmental studies (Kelcey and Lucieer, 2012a). Some of the UAS multispectral sensors, such as multiple camera arrays form Tetracam MCA family or sensors based on converted cameras are implemented using only basic or no calibration protocol and the sensors' radiometric properties are not known (Aasen et al., 2015;Tetracam Inc., 2015). For newest multiple array systems (MicaSense RedEdge and Parrot Sequoia), the basic radiometric calibration is implemented into photogrammetric software (Parrot., 2017). However, such approaches are usually simplified, and for an accurate data collection, analysis, and comparison of data across different sensors, there is a need to develop user-designed correction methods (Assmann et al., 2018). The results of imaging should be processed quickly. * Corresponding author Hence, the used sensor corrections and radiometric calibration of the images should be relatively simple and standardized preferably featuring empirical, image-based calibration methods. The calibration workflow for multiple array multispectral cameras (e. g. Tetracam μMCA, Parrot Sequoia and MicaSense Red-Edge) generally involves: a noise reduction, a vignette effect reduction and an atmospheric correction to surface reflectance using empirical approach (Nocerino et al., 2017;Padró et al., 2019Padró et al., , 2018Pozo et al., 2014). Assmann et al. (2018) proposed workflow of vegetation monitoring with radiometric calibration using Parrot Sequoia. Iqbal et al. (2018) and Guo et al. (2019) successfully evaluated the suitability of the empirical line method for atmospheric corrections of UAS data. The common image-based sensor correction methods are a Dark offset subtraction method for noise reduction (Kelcey and Lucieer, 2012b;Mansouri et al., 2005) and Look-Up Table  (LUT) method for the vignetting correction (Yu, 2004). The main principles of both methods are calculation of a database of correction images in the laboratory and then applying the correction images that were taken under in the same conditions as during flight (Minařík et al., 2019). The large database of correction pictures for all combinations of relevant exposures, ambient temperatures and light intensities under laboratory conditions should be built, which is time and storage consuming. An alternative to this approach is to acquire a set of correction pictures in the field. Dark images could be taken in a portable black box and vignetting correction images could be generated by capturing photos of spectrally homogeneous panels under the same illumination in the field. However, only a few authors have addressed this for hyperspectral sensors. Aasen et al. (2015) and Yang et al. (2017) included in-situ LUT flat field calibration into their processing workflows. The aim of this study is to compare two possible radiometric calibration approaches of Tetracam μMCA Snap multispectral camera using the Dark offset subtraction and LUT methods. We compared a laboratory-based calibration using correction images taken in the laboratory under the controlled conditions with a rapid in-situ based calibration featuring correction images taken during the field campaign. The hypothesis was that the accuracy of in-situ calibration could be comparable with the laboratory calibration, and thus could replace it and simplify the radiometric calibration process. This study stems on and extends the previous study (Minařík et al., 2019) which proposes the calibration workflow for multiple array camera arrays based on laboratory experiments.

Multispectral camera array
We used a Tetracam μ-MCA Snap 6 ( Figure 1) multiple camera array system to capture the images in six independent channels with changeable bandpass filters. The nonuniform global snap sensor's relative sensitivity to light and the nonuniform filter transmittance is compensated using the relative exposure settings to the master channel (Band 4), which are set in the cameras' production and differ for each camera model. The sensor parameters are given in Table 1.   (Kelcey and Lucieer, 2012b;Mansouri et al., 2005) for noise reduction. This method estimates the noise of sensor from dark images in the absence of light when no signal is generated, and only noise is present. Through averaging, the characteristics of the sensor-specific perpixel distribution of the noise can be extracted for the same exposure times that are used in the field. The Standard Deviation (SD) is the approximation of the remaining noise following the dark offset subtraction.

Vignetting reduction
We used look-up table (LUT) correction method (Yu, 2004) for the per-pixel vignette effect, nonuniform quantum efficiency effect, and dust defects reduction. The LUT was calculated from a set of flat field noise corrected images under uniform illumination. The LUT correction method is constructed on the assumption that the brightest pixels in the center are not affected by any error and they can serve as reference values for normalization (Minařík et al., 2019). The correction coefficients were computed as the ratio of the central DN to the DN of each pixel using equation 1 (Yu, 2004): where ILUT(i, j) is the correction coefficient of the pixel at position (i,j), Iref, max is the maximum brightness value of the image and Iref (i, j) is the brightness value of pixel position (i,j).

Atmospheric corrections method
We chose the empirical line method (Teillet, 1986), which is widely used for UAV imaging (Iqbal et al., 2018). The method assigns a reflectance value to each pixel that is based on the computed linear relationship between the known reflectance value of the light and the dark reflectance panels (see Figure 2) and their DN numbers that were extracted from images.

Laboratory calibration
The laboratory sensor calibration of the camera was conducted in the air-conditioned calibration laboratory of the Global Change Research Institute of the Czech Academy of Sciences featuring a CSTM-LR-20-M optical integrating sphere (Labsphere Inc.). Firstly, the sensor´s linear response was investigated in the integrating sphere as a presumption of using an empirical line method for different exposure times (50, 250, 500 and 1000 μS) typically used in the field. The linearity was confirmed using a regression coefficient that was higher than 0.99 regardless the exposure time and light intensity. The dark offset images were taken in a completely dark room using the same exposure times (50, 250, 500 and 1000 μS). Twenty dark offset samples were taken to eliminate a random noise. The average images were calculated. The appropriate dark offset (500 μS) image was subtracted from a field image. The per-pixel vignette effect was corrected using a set of flat field noise corrected images. In the laboratory, the integrating sphere served as the flat field surface. The same datasets were used for calculating vignetting correction coefficients as for linearity testing to be consistent. The average image was computed using the set of 20 images with the same exposure time. After subtracting the corresponding dark offset image from the average flat field image, the correction coefficient images were computed. The appropriate correction image (500 μS) was then applied to the test image from a field experiment. The noise corrected average flat field image with exposure time 500μS was used for Signal-to-Noise Ratio (SNR) estimation. SNR was estimated as the ratio of average DN value extracted from flat field image to the SD of the corresponding dark offset imagery.

Rapid in-situ calibration and field experiment
The field experiment was conducted in March 2018 at noon from a height of 15 m with a Ground Sampling Distance of 0.75 cm ( Figure 2). As a test field, we used a beach volleyball court since the sand represents an almost lambertian surface with stable reflectance (Baugh and Groeneveld, 2008). The exposure time was set to 500 µS in coherence with the illumination and a general rule of thumb that the brightest object on the surface (white target) should reach a maximum of 80% of the radiometric resolution. The rapid in-situ calibration was performed during a field experiment. The portable black box was used for taking the dark images. Twenty dark offset samples were taken to eliminate a random noise. The same exposure time 500 µS was used. The average image was calculated and subtracted from the field image. The calibration target Cream 1 served as the flat field surface for in-situ vignetting corrections. The average image was computed using a set of 20 calibration images from a near-ground level (1 m). After subtracting the corresponding dark offset image from the average flat field image, the correction coefficient image was computed. The field image was multiplied by the correction coefficient image then. Before the accuracy of in-situ and laboratory calibration comparison using validation targets, the effect of vignetting correction method was investigated at selected profile (see Figure  2), because both the vignette effect reduction and non-uniform quantum efficiency of the chip cells have the most significant impact on sensor calibration overall accuracy (Lebourgeois et al., 2008). We compared Coefficients of Variation (CV) of DN extracted from the raw image and corrected image using two proposed calibrating approaches.
The accuracy of laboratory and in-situ radiometric calibration was assessed by comparison of three validation targets reflectance values extracted from corrected UAV images based on laboratory and in-situ calibration with a reference spectroscopy measurement. The calibration targets were placed in the middle to reduce the vignette effect and aid empirical line construction. Three verification targets were placed on the edges and in the corners of the image where the vignette effect was the strongest.
The targets were made of metal plates (40 × 40 cm) that were professionally painted using NEXTEL colours. We computed the empirical line for every channel of the laboratory and in-situ calibrated images extracting values from 500 pixels. The median value was used to assign the reflectance value. We used 500 reflectance values from each validation target for descriptive statistics, Normalized Root Square Error (NRMSE) calculation and statistical testing.

Signal to noise ratio
The results of signal-to-noise analysis are in Table 2. Band 6 records the worst response to dark offset subtraction resulting in the highest remaining noise (SD) and the lowest SNR. This may be due to fact that band 6 has the longest relative exposure time and the lowest filter transmittance. Both properties increase the image noise (Al-Amri et al., 2010). Band 1 records the lowest remaining noise, but the SNR is relatively low. This is due to relatively low dynamic range of flat field image caused by short relative exposure time of band 1. The relative exposure time of band 1 may seems underestimated, but the reason is the radiance difference in the halogen light of the sphere and sunlight. While sunlight has a maximum radiance of approximately 500 nm, the sphere light has a maximum radiance of 900 nm (Minařík et al., 2019). On the other side, the band 4 (master channel) and band 2 generate the highest quality data. It confirms the appropriate master channel selection by manufacturer. The manufacturer selects master channel individually according to the featured set of bandpass filters. The filter that produces the greatest numbers of electrons in response to the radiation is then by the manufacturer selected as master channel (Tetracam Inc., 2015). The other bands reveal balanced SNR. The resulting SNR are higher to values presented by Kelcey and Lucieer (2012a), but it can be caused by different type of surface (artist´s canvas). Hruska et al. (2012) calculated in-flight SNR values around 120 compared to SNR around 250 reported by the manufacturer for hyperspectral sensor PIKA II. This decrease of SNR can occur, because the true SNR is obtained only when calculating a uniform image, as in an integrating sphere or an otherwise uniformly lit featureless image. When the calculation is applied to an image with significant features and luminance variations it reflects only the variability or complexity of the image (Heinold, 2017).

Rapid in-situ and laboratory calibration comparison
The effect of vignette effect reduction was first investigated using the randomly selected sand profile to compare the normalized DNs of the raw and corrected images (Figure 3). The original profile (red lines) reveals a substantial decrease of the DNs that is caused by the vignette effect. Conversely, the normalized DNs show no decrease or only a slight decrease. Moreover, the green in-situ line is almost identical to blue laboratory line. It indicates that both averaged flat field images of integrating and target Cream 1 provide relatively equal options for vignetting correction by LUT method. If we compare the CV of the profiles, we can see that there is a significant reduction in the variation from 50% to 70% after correction occurs in both cases (Table 3).   Table 3. Coefficients of variation at selected profile.
The vignetting correction using the LUT method, which is the most for precise application compared to the optical modelling method (Yu, 2004), significantly reduced the diagonal profile variance. The CV decreased by half in all bands and the DNs were equalized. The similar results were achieved in the study by (Olsen et al., 2010), who presented a calibration of a spaceborne linear scanner using a CCD chip. Moreover, we obtained equal CV values for laboratory and in-situ calibration, which can be considered a simple form of validation. Figure 4 compares the three validation target reflectance curves that are derived from the spectrometer measurement using the median reflectance values of the same targets that are extracted from the raw (red dots) and corrected data using correction images taken in the laboratory (green dots) and in-situ (dark green dots). The most significant improvement occurs as expected at target 4 (a) due to the strongest vignette effect resulting from its location in the selected field image. The raw values are strongly underestimated (~0.35) compared to the reference and corrected values (~0.42). Lower, but still inconsiderable, improvement of the conformity with reference data was obtained at target 7 (c), because of the vignette effect reduction. On the contrary, target 5 (b) reveals only minor improvement, because the target is placed closer to the image center and the vignette effect is not so strong. If we compare the normalized root mean square errors (NRMSEs) between raw data, laboratory and in-situ corrected images (Table 4). We see the similar accuracy improvement of using both calibration approaches. The NRMSE values for target 4 are ten times lower in the corrected images. The NRMSE values of target 7 are 2-10 times lower in the corrected images. The calibration accuracy improvement at target 5 is only minor, because the local vignette effect is relatively minor resulting in low vignetting correction coefficient.  Table 4. The normalized root mean square errors of reflectance values comparing the reference spectral curve of verification targets with reflectance values extracted from the raw and fully corrected images. The normalization was performed by dividing the RMSE by interquartile range. The values are expressed as percentage. WL is wavelength in nanometres, R is raw data, L is laboratory calibrated data and S is in-situ calibrated data. The numbers define the calibration target. * marks lower NRMSE value in the corresponding pair of L and S.
The lower values of laboratory-based calibration NRMSE indicate less residual variance. Hence, we used a Mann-Whitney U test to test the hypothesis that the resulting reflectance values of three validation targets extracted from the laboratory calibrated and in-situ calibrated image are from the same distribution (  Richter and Schläpfer (2002) consider NRMSE values under 2% as a very good agreement with reference data and NRMSE values under 5% as a sufficient match. Moreover, Aasen et al. (2015) implemented in-situ calibration into main image processing chain, because the results were comparable with laboratory calibrations and in-field calibrations perform better for the systematic pattern removal. These findings make the proposed in-situ approach usable for various environmental studies featuring UAV multispectral photogrammetry. To achieve the proper spectral image properties and retain the consistency across multispectral UAV imaging campaigns, the radiometric calibration procedure should be included into every processing chain (Aasen et al., 2018;Kelcey and Lucieer, 2012a). In the case of multispectral frame cameras, the vignette effect removal has the greatest impact on calibration accuracy (see Figures 3,4 and Table 4), which is consistent with the findings of Lebourgeois et al. (2008). Although the statistical testing revealed slightly better agreement of laboratory calibrated reflectance with reference data, the authors consider the proposed rapid in-situ calibration approach as an alternative to classic laboratory radiometric calibration. The differences between the accuracy of both calibration approaches are minor compared to the raw data. A similar agreement between laboratory and field calibration of a modified colour infrared single-lens reflex camera, but without noise removal. was achieved by Crusiol et al. (2017). The correlation coefficient of r > 0.9 was found for the cross calibration.
The main advantage of the proposed in-situ calibration method is that there is no need to additional equipment investment like the integrating sphere and building and maintaining the database of noise and vignetting reduction images apriori, which is the basic principle of dark offset subtraction and LUT correction methods. The calibration images are taken during the filed experiment under the same weather conditions and the database can be built "on the fly". Every step of the proposed workflow can be used as it is designed for another multiple array multispectral cameras systems with monochrome frame global shutter sensors and linear response to radiation e.g. Parrot Sequoia and MicaSense Red-Edge. In addition to simplifying the radiometric correction approach as is proposed here, the empirical line method could be simplified. Moran et al. (2001) refined an empirical line approach using only one within-scene calibration target for factor retrieval from Landsat-5 TM and Landsat-7 ETM+. A similar approach has been tested by Iqbal et al. (2018) for UAV sensor. If the UAV multispectral camera has a linear relationship between raw DN values and reflectance with intercept close to zero, the empirical line can be constructed using only one light target and zero as dark target. Or a universal empirical equation can be used after performing the corresponding experiment (Guo et al., 2019). However, the universal empirical line is model specific, because of the selected set of filters and cannot be applied to a different set (Assmann et al., 2018).

CONCLUSIONS
This study aimed to test and compare two possible radiometric calibration approaches of Tetracam μMCA Snap multispectral camera using the Dark offset subtraction and LUT methods. We compared the standard laboratory-based calibration using correction images from the prepared database with a rapid in-situ based calibration featuring correction images taken during the field campaign. The hypothesis was that the accuracy of in-situ calibration could be comparable with the laboratory calibration, and thus could replace it and simplify the radiometric calibration process by eliminating the main drawbacks of building the large database correction pictures for all combinations of relevant exposures, ambient temperatures and light intensities apriori. The results of the field experiment showed that both calibration approaches led to significant accuracy improvement compared to raw data. The vignetting correction using resulted in a significant reduction of the Coefficient of variation by half in all bands and overall equalizing the DNs on the diagonal profile. The NRMSEs after processing all corrections ranged from 0.24 to 3.40%. The results show that the calculated reflectance estimates based on laboratory calibration are slightly more accurate, because of the controlled illumination conditions resulting in more balanced flat field correction image. Although the statistical resting revealed slightly better agreement of laboratory calibrated reflectance with reference data, it can be concluded that the accuracy of in-situ calibrated approach is sufficient and can be used as an alternative to classic laboratory approach, because the accuracy improvement quantified by the NRMSE is 2 -10 times better using both calibration approaches compared to raw data than the NRMSE differences between them.