AN OPTIMAL IMAGE SELECTION METHOD TO IMPROVE QUALITY OF RELATIVE RADIOMETRIC CALIBRATION FOR UAV MULTISPECTRAL IMAGES

Radiometric calibration has become important pre-processing with increasing use of unmanned aerial vehicle (UAV) images in various applications. In order to convert the digital number (DN) to reflectance, vicarious radiometric calibration is widely used including relative radiometric calibration. Some UAV sensor systems can measure irradiance for precise relative radiometric calibration. However, most of UAV sensor systems cannot measure irradiance and therefore precise relative radiometric calibration is needed to produce reflectance map with vicarious calibration. In this study, an optimal image selection method is proposed to improve quality of relative radiometric calibration. The method, relative calibration by the optimal path (RCOP), uses filtered tie points acquired in geometric calibration based on selection optimal image by Dijkstra algorithm. About 100 multispectral images were acquired with a RedEdge-M camera and a fixed-wing UAV. The reflectance map was produced using RCOP and vicarious calibration using ground reference panels. A validation data was processed using irradiance for precise relative radiometric calibration. As a result, the RCOP method showed root mean square error (RMSE) of 0.03-0.10 reflectance to validation data. Therefore, the proposed method can be used to produce precise reflectance map by vicarious calibration. * Corresponding author


INTRODUCTION
Rapidly advancing technologies are making sensors smaller, more precise, and more popular. Recently, the supply of UAV images is increasing in various remote sensing fields. UAV images have attracted attention as a means of efficiently observing limited access areas when users want them (Tsouros et al., 2019). UAV images are used to acquire three-dimensional spatial information and biophysical information. In order to acquire location and attribute information from remote sensing images including UAV images, pre-processing such as geometric and radiometric calibration is required. Radiometric calibration, including atmospheric correction, is an important pre-processing step to obtain biophysical information from spectral reflectance. For radiometric calibration, vicarious calibration and the radiative transfer (RT) model are widely used in spaceborne and airborne remote sensing fields (Del Pozo et al., 2014;Moran et al., 2001;Smith, Milton, 1999;Berni et al., 2014;Garzonio et al., 2017;Honkavaara et al, 2013). The RT model converts DNs into reflectance based on a mathematical model. It does not require ground reference panels, but it has limitations such as complicate parameters and low accuracy (Yang et al., 2017;Hakala et al., 2013;Aasen, Bolten, 2018;Konkavaara, Khoramshahi, 2018;Roosjen et al., 2017;Schneider-Zapp et al., 2019). UAV images are taken at a relatively low altitude, compared to aerial or satellite images, and they may do not possess significant radiometric distortions. On the other hand, their small field of view makes mosaicking a necessary procedure. Each image may experience different turbulence, a different incidence angle, different illumination, or different signal processing chains. As a result, radiometric properties of UAV images may vary significantly. Relative radiometric properties among UAV images have to be adjusted so that the mosaic image can have a consistent spectral reflectance. Therefore, for UAV images, we need relative radiometric calibration in addition to vicarious calibration, unless we install ground reference panels within the field of view of each image. Images with ground reference panels are calibrated through vicarious calibration. Images without ground reference panels are calibrated relatively to the DNs of the images with ground reference panels, and are then calibrated through vicarious calibration (Mafanya et al., 2018). Some UAV cameras include an irradiance sensor to measure the amount of sunlight for each image, and they use the irradiance measurements for relative radiometric calibration (Mamaghani, Salvaggio, 2019). For UAV cameras without an irradiance sensor, conversion coefficients have been estimated through regression analysis between the DNs of pixels on overlapping regions between two images (Suomalainen et al., 2018). However, this method is prone to geometric distortions and pixels with radiometric anomalies (Xu et al., 2019), and they often result in visual discontinuity between adjacent scenes (Liu et al., 2011). Precise relative radiometric calibration of UAV images is still highly in demand. In this study, an optimal image selection method is proposed to improve quality of relative radiometric calibration. It uses filtered tie points acquired in geometric calibration based on optimal image selection by Dijkstra algorithm. It can minimize error accumulation by reducing the step of reaching the last image located at the region boundary.

UAV Images
UAV images were acquired from 15:32 to 15:36 on 15 May 2019 with 50 degree of solar elevation angle under clear sky. Total 108 images were acquired with about 3 cm spatial resolution. Figure 1 shows the study area and locations of the images acquired. The area is above the campus of the National Institute of Agricultural Sciences in Wanju-gun, Jeonlabuk-do, Korea. It includes trees, grass, soil, and sidewalk blocks as its major cover types. The camera captures images in five spectral bands: blue, green, red, red-edge, and near-infrared. It also has an irradiance sensor, a down-welling light sensor (DLS), for radiometric calibration. The measured irradiance from the DLS was used to produce validation data for this study.

Reflectance of Ground Reference Panels
Seven ground reference panels were deployed on the grass of a soccer field for vicarious radiometric calibration of the UAV images ( Figure 3a). The panels were made of specially coated fabric sized 1.2 m × 1.2 m to maintain constant reflectance through a spectral range of 435 nm to 1100 nm. Spectral reflectance of each panel was measured using a FieldSpec-3 spectro-radiometer ( Figure 3b). The panels used for the experiment provided 3%, 5%, 11%, 22%, 33%, 44%, and 55% reflectance.
(a) (b) Figure 3. (a) Ground reference panels installed for vicarious radiometric calibration of UAV images, and (b) spectral reflectance measured using a spectro-radiometer

Validation Data
For validation, a reference reflectance map was generated using the measured irradiance data and the reference panels. An image map with intensity as its pixel values was generated first through standard processing procedures of tie point extraction, bundle adjustment, digital surface model generation, orthorectification, and image resampling. A reflectance map with reflectance as its pixel values was then generated by converting DNs of the mosaicked image into reflectance values. For the conversion procedure, the measured irradiance data and the DN on the ground reference panels were used to calculate coefficients for the conversion. We used commercial software (SW) to produce the reflectance map. Figure 4 shows the reflectance map generated as validation data.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2020, 2020 XXIV ISPRS Congress (2020 edition) Figure 4. A natural color composite of reflectance map generated for validation 3. METHODS

Relative Radiometric Calibration Procedure
Overall radiometric calibration procedure is shown as Figure 5 to generate reflectance map. The first procedure is vicarious radiometric calibration of a reference image using ground reference panels. The image with ground reference panels is selected as the reference image. DN values of the reference panels within the reference image are measured, and coefficients for vicarious calibration are estimated through regression analysis. Next, an image network is formed by defining each image as a node. When there is a sufficient number of tie points between two images, a link between the two corresponding nodes is defined. After an image network is formed, we can find an optimal path from one image to another by following links between image nodes. In this experiment, we used the Dijkstra algorithm to obtain the optimal path. Next, tie points among the images within the optimal path are processed to estimate coefficients for relative radiometric calibration. DNs of an image are converted to equivalent DNs in the reference image using these relative calibration coefficients, and eventually, they are converted to reflectance using the vicarious calibration coefficients. Finally, a geometric mosaicking process is carried out on the reflectance images to generate a mosaicked reflectance map. Figure 5. Overall procedure of radiometric calibration

Vicarious Calibration of Reference Image
DNs of the reference image are converted to spectral reflectance using the vicarious radiometric calibration method. For each ground reference panel, its image location is identified manually. The average DN around the location was calculated per spectral band and per reference panel. DNs of the reference image are converted to reflectance using the following equation: (1) where R is reflectance, DN is the digital number of an image, is absolute radiometric gain, and is the absolute radiometric offset. These coefficients are estimated for each band via linear regression between the DN and the reflectance of the pixels corresponding to the ground reference panels ( Figure 6). All DNs of the reference image were then converted to reflectance using equation (1).
(a) (b) Figure 6. Examples of linear regression analysis between the digital number and the reflectance for (a) blue and (b) red-edge bands

An Optimal Image Selection Method
A method to select optimal images was proposed that is relative calibration by optimal path (RCOP). It can minimize error accumulation by reducing the step of reaching the last image located at the region boundary. Yin and Wang (2010) proposed Dijkstra algorithm to find a path that minimizes the sum of The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B1-2020, 2020XXIV ISPRS Congress (2020 weights from one starting point to all other points. In Figure 7, the weight between images was given as 1 when number of tie points was exceeded 100, otherwise it was given as infinity. The reason is that if the number of tie points between images is too small, it may cause over-or under-estimation of conversion coefficients. If there are multiple paths with the weight, the optimal path is selected in the direction where the number of tie points is greater. Figure 7. Selection of optimal images using number of tie points between image pair that was selected by weight and number of tie points

Relative Radiometric Calibration
Tie points are obtained from the geometric calibration process. Some tie points that may be geometrically correct may contain radiometric abnormalities due to shadow, saturation, or sun glints. To improve the quality of relative radiometric calibration, tie points undergo radiometric filtering. In order to remove radiometric outliers in the tie points, the statistical distribution of DN differences between two images is analyzed. The mean and standard deviation of the DN differences are calculated. We accept tie points where the DN differences between two images are within range of the mean ± standard deviation. Figure 8 shows an example of the distribution of DN values for all tie points (black crosses), and the accepted tie points are red crosses.

Figure 8. The distribution of DN values for all tie points (black crosses) and accepted tie points (red crosses)
Using the accepted tie points, coefficients of relative radiometric calibration are estimated using equation (2): (2) where DN is the digital number from the original image, DN' is the digital number from the converted image, ar is relative radiometric gain, and br is relative radiometric offset. Using the estimated conversion coefficients, DNs of the original image are converted to equivalent DNs of the other image. Using successive image pairs along the optimal path, DNs of the original image are converted sequentially to equivalent DNs of the reference image. They are then converted to reflectance using equation (1).

Visual Interpretation of Calibration Results
Figures 9 shows the natural color composite of the reflectance map by RCOP and validation data, respectively. In order to compare colors, each band was stretched based on the same range of reflectance. The results from RCOP had similar color when compared with the validation data using irradiance measurement. The mosaic result was smooth without noticeable color differences at the boundaries between images.

Quantitative Accuracy Analysis
For quantitative validation, reflectance obtained from RCOP was compared to the validation data at the same points. The test samples were collected for a total of 200 pixels by random sampling (Figure 10). Error was calculated as root mean square error (RMSE) using the following equation: ( 3) where n is the number of samples, and and represent reflectance of the calibrated image and the validation data, respectively, for the i-th validation point. The RMSE of the visible (blue, green, red) bands is about 0.03, although red-edge and NIR bands show about 0.06 and 0.10, respectively. A reason of higher RMSE of red-edge and NIR bands might be cover types of study area that most of the area is composed with grass and trees. Vegetation has higher reflectance in red-edge and NIR bands than visible bands.

CONCLUSION
In this study, the RCOP method was proposed for relative radiometric calibration of images those do not have measured irradiance. It can improve radiometric calibration quality using tie points and optimal path selection. It showed reliable and stable calibration results. Therefore, the proposed method can be used to obtain a precise reflectance map, improving the quality of relative radiometric calibration. Most UAVs acquire images without irradiance measurement, and are used in applications where precise reflectance retrieval is crucial. The proposed method should contribute to improving accuracy of biophysical factor estimation or classification using UAV images. In further studies, precise band alignment will be carried out, and we will check how exceptions reported in this paper can be improved. An additional validation study will be carried out by installing ground reference panels at various locations in the study area.