RADIOMETRIC QUALITY ASSESSMENT FOR MAXAR HD IMAGERY

The requirement for very high-resolution satellite imagery by different applications has been increasing continuously. Several commercial and government-supported missions provide sub-meter spatial resolutions from optical sensors aboard Earth Observation (EO) satellites. The MAXAR satellite constellation acquires images with up to 30 cm Ground Sampling Distances (GSDs); and the High-Definition (HD) image production technology developed by MAXAR doubles the resolution by using artificial intelligence methods. Although the spatial resolution is one of the most important image quality metrics, several other factors indicated by diverse radiometric and geometric characteristics may circumscribe the usability of data in different projects. As part of mandatory activities of European Space Agency (ESA), Earthnet Programme provides a framework for integrating ThirdParty Missions into the overall EO strategy and promotes the international use of the data. The Earthnet Data Assessment Pilot (EDAP) project aims at assessing the quality and the suitability of TPMs, and provides a communication platform between mission providers to ensure the coherence of the systems. In this study, the radiometric quality of the MAXAR HD products was evaluated within the EDAP project framework by using several General Image-Quality Equation (GIQE) metrics, visual inspections, and comparative assessments with orthophotos obtained from an Unmanned Aerial Vehicle (UAV) platform and with the original (nonHD) orthophotos with 30 cm resolutions. The results show that the spatial resolution improvements are observable in urban areas, where sharp edges are present. However, blurring and color noise patterns also occured in the HD images. * Corresponding author


INTRODUCTION
The variety of applications using satellite optical imagery with very high spatial resolution at sub-meter level is increasing each day, such as agriculture (Zhang et al., 2020), land use land cover (LULC) classification (Zhang et al., 2018), 3D surface and building modeling (Poli et al., 2015), ecosystem modeling (Gruen et al., 2017), etc. As part of radiometric quality improvement efforts, satellite image vendors such as MAXAR Technologies aim to generate image products with even higher spatial resolutions by applying machine learning algorithms (Blog MAXAR, 2021). The MAXAR satellite constellation acquires images with a nominal Ground Sampling Distance (GSD) of 30 cm in an area of approximately 680,000 km² per day. The High-Definition (HD) image production technology developed by MAXAR increases the image GSD to 15 cm by using artificial intelligence methods (Blog MAXAR, 2021). The HD technology can be applied not only to 30 cm but also to 40 cm -60 cm resolution images (European Space Imaging, 2021). The HD products are output of an image processing algorithm, and therefore questions can be raised regarding the radiometry, and how the image content is preserved and/or transformed.
Considering the diversity in Earth Observation (EO) satellite missions and the specifications of the sensors owned by different agencies and commercial bodies, the need of efforts for constituting a platform to strengthen the communication between various mission providers and to ensure the coherence of the systems is indisputable. The Earthnet Programme of European Space Agency (ESA) has played a significant role by contributing to the Global Earth Observation System of Systems (GEOSS) for over 40 years . For this purpose, a framework was provided for coordinating non-ESA, i.e. Third Party Missions (TPM), into the overall ESA EO strategy. Therefore, ESA Earthnet Data Assessment Pilot (EDAP) project aimed at carrying out early data quality assessments for various missions, which eventually become TPM, by covering sensor-specific calibration and validation tasks . Leachtenauer et al. (1997) considered three quality attributes for images in the General Image-Quality Equation (GIQE), such as scale expressed in GSD, sharpness measured from Modulation Transfer Function (MTF) and the noise level as Signal-to-Noise Ratio (SNR). For the interpretability of sharpness, Relative Edge Response (RER), which perceives sharpness or acutance, is also evaluated. The SNR definitions vary in different remote sensing systems (Fiete and Tantalo, 2001). Pagnutti et al. (2010) reviewed the image quality parameters and included Point Spread Function (PSF), Line Spread Function (LSF), MTF, Contrast Transfer Function (CTF), and Edge Spread Function (ESF) as frequently used ones. Valenzuela and Reyes (2019) considered six criteria, i.e. GSD, Rayleigh diffraction limit, ground spot size, generalized Rayleigh resolution criterion, sparrow limit, and the Full-Width at Half-Maximum (FWHM) of the PSF/LSF for the radiometric quality evaluation of satellite sensors at detector, optics and system levels. On the other hand, Cenci et al. (2021) proposed a sharpness assessment algorithm based on natural edges; distinguished the MTF values for the system and the end-user product; and concluded that the method must be evaluated with respect to the nominal GSD of a sensor, image pixel size, and the SNR.
In this study, the radiometric quality of MAXAR HD products with 15 cm spatial resolution was evaluated within the EDAP project framework by using several image metrics and visual comparisons in different test sites, such as; Comparison of HD images with the optical images acquired from an Unmanned Aerial Vehicle (UAV) platform in a specially designated satellite Calibration/Validation (Cal/Val) testfield in a part of Ankara, Turkey.
(ii) Evaluation of GIQE metrics (Leachtenauer et al., 1997) for HD images including MTF, RER, and SNR by using an MTF target located in Salon-de-Provence, France and natural targets selected in different datasets.
(iii) Performing comparative assessments between the spatial resolutions (30 cm) of the originally generated orthophotos products and the HD data generated in a postprocessing through visual inspections and histogram evaluations over Munich, Germany, and Al Jufrah, Libya.
The sites were selected based on reference availability (i.e. UAV data, MTF target, suitable natural targets), LULC type (e.g. airport, urban structures, desert), and the provision of 30 cm and 15 cm images. The methodology, the study datasets and the results are presented and discussed in this paper.

METHODOLOGY
In this study, the system MTF together with PSF and LSF were estimated since they are crucial to characterize the image contrast. A classical slanted-edge (i.e. knife-edge) method was applied (Viallefont-Robinet et al., 2018;Kohm, 2004;Crespi and Vendictis, 2009). The implementation of the slanted-edge method has four steps, i.e. edge identification and orientation, edge modeling, ESF construction, and MTF computation from the ESF (Viallefont-Robinet et al., 2018). The slanted-edge method used in the study was inspired from the method of Kohm (2004). The ESF modeling is a crucial processing step and the methods can be divided into two main categories: modeling based on a parametric and on a non-parametric approach. The two approaches were systematically tested, and in most cases the parametric approach was used as discussed in (Leloglu and Tunali, 2006) since it is attractive, with the risk of loosing fidelity in the system modeling. In the context of MAXAR HD, it was possible to use a non-parametric approach. Even if less robust to noise, it is more convenient to analyze deformation sourced from the pan-sharpening process.
The MTF analysis is usually performed by using a specific (check board) artificial target for calibration, whilst natural targets are mostly used for validation. An interest of the MTF analysis is also the deformation of the PSF across field of view, which is not possible with an artificial target. The MTF artificial target is located in Salon-de-Province, France. As part of natural target suitable to VHR data, a number of natural targets considering suitable image edges at sharp transition were selected and processed. As generally performed in the community, we start from the assumption that the product MTF is separable. Also, MTF can be fully characterized by analyzing the values in along-track and across-track directions.
The data was in the form of pixel intensity values (i.e. digital number -DN) obtained after radiometric correction, and the top-of-atmosphere (TOA) radiance values were calculated using Equation 1 (MAXAR, 2021) for MTF and SNR analyses.
where L = top-of-atmosphere radiance DN = pixel value GAIN, OFFSET = the absolute radiometric calibration band dependent adjustment factors abscalfactor, effectivebandwidth = TDI-specific parameters delivered in the IMD file (Maxar, 2021) Besides the MTF, the image noise was investigated by computing the SNR and analyzing histograms of image parts selected in homogenous surfaces (natural targets). Although such analysis can be performed in both homogeneous and nonhomogeneous areas; here, the analyses were carried out in homogeneous areas; since the image texture differences affect the noise analysis in non-homogeneous images. In addition, the mean and the standard deviation values calculated by using the DN values in selected image patches over water surfaces were used for image noise analysis as proposed by Baltsavias et al. (2001). A 5x5 kernel window was used here; and the statistical values were computed in a stepwise manner. 70% of the results obtained from all kernels were included in the statistical analysis based on the assumption that 30% with largest standard deviations could be outliers.

The Study Datasets
Absolute radiometric calibration of MAXAR imagery have regularly been carried out and the results were evaluated using images with < 20-deg off-nadir angle (Kuester and Ochoa, 2019). The RadCalNet-based evaluations show that the data is within specification and the sensors are radiometrically stable. However, large Off-Nadir View Angle cause variations due to surface Bidirectional Reflectance Distribution Function (BRDF) and longer atmospheric path, which are up to 10% (Kuester and Ochoa, 2019). According to the metadata files (.IMD) provided with each product, radiometric correction was applied to all images used here. The correction deals with artefacts that are not sourced from image objects or scene; instead sensor errors, i.e. non-responsive detectors, scanner inconsistencies, and atmospheric interference (DigitalGlobe, 2014).
All MAXAR datasets evaluated here were obtained from the Worldview-3 (WV-3) satellite. In Table 1 (1) The The UAV data employed here was acquired on 11 Jan 2021 between 10-11 a.m. UTC with a Sony RX1 RII RGB camera. The average GSDs of the images is 3.25 cm. A digital surface model (DSM) and an orthophoto mosaic were produced after performing bundle block adjustment with six Ground Control Points (GCPs). The orthophoto of the green band was downsampled to 15 cm using the nearest neighbour method to match the MAXAR HD resolution. The location of the Ankara testfield, a part of the MAXAR pan band data used for the evaluations, the UAV orthophotos and the surface characteristics of the study area (i.e. DSM) are presented in Figure 1.

MTF and Noise Analysis Results
The MTF evaluations and SNR computations were performed by using the MTF target located in Salon-de-Provence using the pan HD image (Figure 2a); and two natural targets, such as a building roof in Munich and runway signalization in Gibraltar Airport, both from green band images with 15 cm resolution (Figure 2c-d). As can be seen in Figure 2a, the target has some color deformations. Therefore, the calculations were performed several times at different lines to obtain the edges without deformations. The ESF, LSF, FWHM, SNR, RER and MTF were calculated in four directions, i.e. two Along-Track (AL) and two Across-Track (AC) as shown in Figure 2b. The SNR, RER, FWHM and MTF values obtained from each direction (Hh, Hl, Vh, Vl) are presented in Table 2. The graphs depicting ESF, LSF, FWHM, SNR, RER and MTF results obtained from the target are presented in Figures 3-6 again separately for each direction. The results obtained from the natural targets are presented in Table 2 and Figures 7 and 8. It must be noted that although the target in Gibraltar airport has a narrower radiometric range than the dynamic range of the sensor, it was used here due to the high contrast and the availability of the suitable along-and across-track edges. According to Cenci et al. (2021), the FWHM provides a more robust estimation of sharpness than MTF for representing the spatial resolution. In addition, the FWHM values ranging between 1-2 indicates images with suitable sharpness, whereas values above 2 present blurry effects in the images. Considering the FWHM results in  The noise estimation for the pan band was carried out over a dam reservoir in Ankara by evaluating a total of 9216 kernel windows with a size of 5 x 5. The DN range of the image part was between 123-163. The mean of the standard deviation values obtained from the 70% of the kernels (6451 in total) was 1.99 pixels with a standard deviation of 0.35. The noise was comparatively analyzed over a lake in Munich with 30 cm and 15 cm resolution green band data by using 6451 kernels (70% of the initial) as in Ankara. The mean of the standard deviations obtained from the 30 cm and the HD data were 3.40 and 2.08 pixels, whereas the standard deviations from the mean was 0.41 and 0.40, respectively. The results indicate that the HD data has lower noise level for the green band.

UAV Comparison Results
The UAV-MAXAR HD data were compared visually by using the pan HD data of WV-3 and the green band of the UAV orthophotos downsampled to 15 cm resolution. Selected image patches are presented in Figure 9. The radiometric resolutions of the WV-3 and the UAV data are 11 bit and 8 bits, respectively. The images were displayed with a min-max scaling in Quantum GIS (QGIS) software. The patches were selected from a graveyard, a playground, railway, and water body. The histograms of the patches over the water surface are also provided in the Figure. From the Figure, it can be seen that the level of detail that are recognizable in the UAV images is superior to MAXAR. Here, the differences source from several factors, such as the distance between the object and the sensor, and the athmospheric and the illumination conditions. In addition, the Mean Off-Nadir View Angle of MAXAR data was relatively high (21.9). As can be seen over the water surface, the WV-3 image exhibits uniform random Gaussian noise.

HD versus Original Ortohophoto
The evaluations were performed using selected image patches in Munich and Al Jufrah, since both the HD and the original orthophotos were provided for these regions. In addition, Munich is an urban area and Al Jufrah has desert areas with high reflectances. The Munich patches in RGB color combination given in Figures 10 and 11 were selected over railway, soccer field, road, and water surface. From the railway, it can be seen that the edges are sharper with 15 cm (Figure 10). In the soccer field, aliasing was observed in the 30 cm data and color deformations and blurring were observable with the HD data. The histograms of both images show that the higher resolution one has smoothed histogram graphs. Bands 1-2-3-4 are blue, green, red and near-infrared in the Figure. The trucks and cars observed on the road in Figure 10 also show that the 15 cm processing yields to noise in color. The water surfaces, which were also displayed with a stretching to min-max values in Figure 11, and their respective histograms show that a smoothing occurs with the HD processing.

UAV (green band)
Min The Al Jufrah patches were selected over four areas, two from the airport runway ( Figure 12) and two from the desert ( Figure  13). The histograms over the two desert areas (Figure 13a-b) are also presented. The airport runway areas given in Figure 12 show aliasing in the 30 cm resolution and blurring in 15 cm, although the edges are sharper. Color deformations were observable also in the 30 cm data on the runway. A noise with square pattern and color deformations are clearly observable in the HD data over the desert.

CONCLUSIONS AND FUTURE WORK
In this study, the radiometric quality including image sharpness and noise of MAXAR HD products were evaluated by using various GIQE metrics such as ESF, LSF, FWHM, SNR, RER and MTF values. The metrics were evaluated both using natural targets and an MTF target over Salon-de-Provence, France. In addition, visual investigations were carried out by comparing the 30 cm and 15 cm (HD) images over Munich and Al Jufrah. Furthermore, visual assessments on the image level of details with the HD pan image and a UAV orthophotos with the same resolution were carried out. The results show that although the edges are sharper in many areas, the pan-sharpening process caused blurring and a pattern noise in the HD images. The blurring was observed also with the FWHM values. The pansharpening also affect the ESF function and cause a bump in the curve. Color deformations were also observed in several patches in high reflectance areas, such as desert. Future work includes the extension of the FWHM, SNR, and MTF analyses by using more natural targets.