Image Quality Assessment of High-resolution Satellite Images With MTF-based Fuzzy Comprehensive Evaluation Method

: A Modulation Transfer Function (MTF)-based fuzzy comprehensive evaluation method was proposed in this paper for the purpose of evaluating high-resolution satellite image quality. To establish the factor set, two MTF features and seven radiant features were extracted from the knife-edge region of image patch, which included Nyquist, MTF0.5, entropy, peak signal to noise ratio (PSNR), average difference, edge intensity, average gradient, contrast and ground spatial distance (GSD). After analyzing the statistical distribution of above features, a fuzzy evaluation threshold table and fuzzy evaluation membership functions was established. The experiments for comprehensive quality assessment of different natural and artificial objects was done with GF2 image patches. The results showed that the calibration field image has the highest quality scores. The water image has closest image quality to the calibration field, quality of building image is a little poor than water image, but much higher than farmland image. In order to test the influence of different features on quality evaluation, the experiment with different weights were tested on GF2 and SPOT7 images. The results showed that different weights correspond different evaluating effectiveness. In the case of setting up the weights of edge features and GSD, the image quality of GF2 is better than SPOT7. However, when setting MTF and PSNR as main factor, the image quality of SPOT7 is better than GF2.


INTRODUCTION
Image quality has great effects on applications of high-resolution satellite data, like visual reconstruction, remote sensing interpretation and production of digital orthophoto map (DOM) (Li L, 2015).How to evaluate the quality of high-resolution remote sensing image scientifically and effectively has become a hot research direction.
Traditional image quality evaluation includes subjective evaluation and objective evaluation.For the subjective evaluation, relevant experts evaluate the satellite image quality through observing the image and concluding the quality score based on certain evaluation criteria.The most commonly used subjective evaluation method is National Imagery Interpretability Rating Scale (NIIRS) (Irvine J M, 1997).On the other hand, objective quality evaluation is mostly achieved by analyzing variable quality indicators of remote sensing image, including noise (Wang Y, 2003), gray statistic, texture (Sun Z, 2010), definition (Wang R B, 2015), MTF (Miao Z, 2014) and so on.
In particularly, MTF is an important indicator of the quality of remote sensing image.It reflects the attenuation of image frequency domain after imaging.The measurement of on-orbit MTF is mostly based on the analysis of edges in the satellite images.The edge based MTF extraction has been added to ISO Standard 12233 (ISO, 2000).The edges are usually extracted from two kinds of surface targets, like artificial architectures and natural surfaces.For example, the MTF measurement of French SPOT was based on the grassy field boundary (Leger D, 2003).
The MTF test for U.S. IKONOS was done by use of an artificial checkerboard target and a target painted with black-and-white bars as fringe objects (Choi T, 2002).In general, MTF is one of the most practical and commonly used indexes for the quality evaluation of remote sensing images.As the knife-edge area of high-resolution remote sensing images is easier to be extracted, MTF is very suitable for the quality evaluation of high resolution images.
Compared to the middle and low spatial resolution images, high spatial resolution satellite images contain more architectural features of natural and artificial objects.The diverse and heterogeneous terrain characteristics bring about challenges in image quality feature extraction.Meanwhile, the haze and mist near the surface results in the visual effect of a loss of contrast in the subject, which could be a main factor of image quality deterioration.In addition, the previous methods for satellite image quality evaluation only use certain indices and lack the comprehensive evaluation of the image spectral and imaging quality.For examples, in the paper (Crespi M, 2009), the feature parameters such as SGD, SNR, MTF were used to evaluate high resolution image quality of Quickbird, WorldView-1 and Cartosat-1 respectively.By using of statistical analysis method, researchers evaluate the characteristics of imagery of IKONOS (Pagnutti M, 2003), ZY-3 (Xu X, 2012), and ALOS (He Y H, 2008) satellites.However, the comprehensive quality evaluation of high-resolution satellite image is rarely studied.Wang Rong Lin proposed a comprehensive evaluation method to evaluate image quality of Landsat TM (Wang R B, 2014).This method used a fuzzy evaluation function to calculate a qualitative evaluation score by fully considering feature indices of gray distribution, information entropy, definition, resolution, noise, cloud, and invalid pixel.The paper showed that fuzzy evaluation can accurately evaluate the quality of remote sensing images with good consistency with human eye evaluation.
In this study, we propose to assess the image quality of highresolution satellite images by using of an efficient fuzzy comprehensive quality evaluation method, and we consider image features like MTF, information entropy, PSNR, average difference, the intensity of edge, average gradient, contrast and GSD.Two experiments were carried out with GF2 and SPOT7 panchromatic images, the results show the robustness and effectiveness of this method, and it can provide reliable reference for quality evaluation of GF2 images.This paper is structured as follows.Section 2 presents the MTF and radiant features extracted from remote sensing images and their impacts on image quality.Section 3 introduces the principle of fuzzy comprehensive evaluation method and how to take use of it to evaluate image quality.Section 4 gives the satellite image information and describes the experimental process.Section 5 presents different experimental settings and results.Finally, the analyses are tied together with conclusions for the proposed MTF-based fuzzy comprehensive evaluation method.

MTF AND RADIANT FEATURES
The spatial and spectral image features are extracted from remote sensing images.They are used to build the fuzzy set of fuzzy comprehensive evaluation function.The MTF represents the frequency characteristics of satellite image, and the radiation features describe the statistics and the spatial variations of images.All of these features reflect the imaging quality from different aspects, and they are extracted from the knife-edge regions of remote sensing image patches.

MTF Features
The value of the modulation transfer function represents the ability of the optical imaging system to redistribute radiant energy (Javan F D, 2013).During the process of satellite imaging, the energy of surface reflection and radiation would attenuate after across the satellite optical imaging system, while using MTF can describe the attenuation of radiation energy from the frequency domain.
In physical optics, the definition of MTF is the ratio of output and input modulation, as Eq.(2-1) shows,   () is the input modulation and   () is the output modulation,  is the frequency.The modulation expresses the brightness contrast of image, as Eq.(2-2) shows.  and   are the maximum and minimum signals brightness respectively.Since the output modulation of the imaging system is always less than the input modulation, the MTF value is between 0 and 1.The larger the MTF, the smaller the difference between image and targets, which means that the performance of the imaging system is closer to perfect.
However, it is difficult to calculate the input modulation, so the researchers proposed several methods to measure the MTF function indirectly, including the point source method (Leger D, 2003), the pulse method (Choi T, 2002), the knife-edge method (Javan F D, 2013) and so on.The measurement of on-orbit MTF is mostly based on the analysis of natural and artificial edges in the satellite sensing images which is easier than other methods, so we use the knife-edge method in this research.Firstly, a knifeedge region should be chosen from the remote sensing image.Then, the accurate edge location is estimated after taking a onedimensional derivative of each row data and finding the centroid to a subpixel accuracy.Next, sub-pixel sampling the edge buffer area.After projecting the sub-pixels onto the gradient direction of the knife edge, the discrete Edge Spread Function (ESF) is obtained.Then, the derivative of ESF yields the line spread function(LSF).Finally, performing a discrete fourier transform, and normalizing, the MTF over a range of horizontal spatial frequencies beyond the Nyquist frequency can be estimated (Maasaoka K, 2014).The calculation results on the calibration field are shown in figure.1.

The Radiant Features
The radiation features used in this paper include information entropy, PSNR, average difference, the intensity of edge, average gradient, contrast and GSD.
1) The information entropy.In remote sensing image processing, the information entropy is often used to measure the dispersion and uniformity of image brightness of ground objects, it represents the disorder degree of remote sensing image.The mathematical expression shows as below. (2-3) Where   is the probability of each gray level appearing in the image, and H is information entropy.
2) Peak signal to noise ratio (PSNR).The PSNR is the important metric which is used to measure the quality of the restored image when it is corrupted due to noise and blur.This metric performs well in high-resolution images.Higher the value of PSNR, indicates higher the quality rate.PSNR can be computed by using the following expression: Where n is the grey level quantization of the image.The MSE is Mean Squared Error, when evaluating an image, the MSE is calculated by image gray and mean value.
Where m and n is the height and the width of image,  ̅ is the grayscale average.
3) The average difference(AD).The average difference is the mean of the difference between adjacent pixels.The texture changes faster as the larger average difference which suggests that the image quality is better.
Where m and n are the width and height of the image.
4) The edge intensity (EI).The edge is one of the most basic features on image and contains important information for image recognition.The human visual system is sensitive to the edges of images.Therefore, researchers often use edge features to describe image quality.The intensity of edge reflects the change of edge point gradient along the edge normal direction.The higher the edge intensity, the better the image quality Sx = I * sobelx (2-7) Where sobelx and sobely are sobel operators in horizontal and vertical direction, * is the convolution operation.
5) The average gradient (AG).Based on the degree of gray change in the fixed direction of image, the average gradient reflects the ability of image detail expression.The higher gradient in the fixed direction, the more obvious gray change, which means that image has better clarity and hierarchy.For discrete image gray matrix, the calculation method of average gradient is shown below. (2-10) Where  is the average gradient,   ′ and   ′ are the first derivatives of horizontal and vertical directions.
6) The contrast.Contrast is the index that reflects the degree of light change.The larger the contrast, the richer the light and dark transition layers of the remote sensing image, the more information about the ground objects can be shown.There many kinds of definitions of contrast, in this research, Michelson contrast is used to represent the contrast feature.
With   and   representing the highest and lowest gray value.
7) Ground Spatial Distance (GSD).GSD represents the maximum distance from which the surface objects can be interpreted in the image, it relates to the extent to which the surface objects can be interpreted.While evaluating the quality of remote sensing images with different resolutions, GSD is one of the critical factors.

FUZZY COMPREHENSIVE EVALUATION
Traditional MTF and PSNR based image quality assessment can only reflect the quality of independent aspects and lack of overall analysis.While the decision-making process for a single feature is fairly straightforward, it is necessary to consider how to avoid one-sidedness when multiple factors are taken into account (Guo L, 2009).By applying the fuzzy comprehensive evaluation, we synthesize various factors to evaluate the quality of a remote sensing image.In addition, because the image quality is fuzzy, one can't tell whether it is good or bad simply (Zhai L, 2007).To get a quantitative analysis results, a fuzzy comprehensive evaluation method is used in this paper.
The fuzzy set theory is the extension of conventional set, Zadeh introduced it in 1965 as a mean to model the vagueness and ambiguity in complex systems (Zadeh L A,1965;Zadeh L A, 1975).And the fuzzy mathematics theory is also applied in the field of quality evaluation.According to the fuzzy mathematics theory, the fuzzy comprehensive evaluation can be applied in three stages.Firstly, the membership function is established based on prior knowledge which is obtained from test data.Secondly, using the membership function, a fuzzy relationship matrix is formed.Lastly, the image quality score is given by fuzzy comprehensive evaluation.The following is the principle procedures of it.

The Factor Set
The factor set consists of various factors affecting the evaluation objective.In this study, nine image features mentioned above constitute a fuzzy set.Assuming that U = { 1 ,  2 ,  3 , ⋯ ,   } is the fuzzy set, each factor   represents the evaluation feature and   can be divided further.Besides, five assessment grades are determined: excellent ( 1 ), good ( 2 ), fair ( 3 ), poor ( 4 ), very bad ( 5 ).The quality grades set V = { 1 ,  2 ,  3 ,  4 ,  5 } is determined by the experimental subject.(3-1)

The Weight Set
In the fuzzy comprehensive evaluation, every evaluation feature has different contribution to image quality.Thus, the weight W = { 1 ,  2 ,  3 , ⋯ ,   } represents the different influence on the decision-making of every image features.The value of weight W is obtained by prior experience and preliminary experiments.

The Membership Function
The shape of membership functions has a direct impact on evaluation results (Jain A, 2016).There are many kinds of membership functions including trapezium, triangle, bell-shaped, and Gaussian distribution, among others.In the determination of the membership function, the membership functions of nine features are defined by triangle distribution in this study.The shapes of the membership functions are shown in the figure below.Where   is the threshold value of image feature.

DATA AND METHOD
Two panchromatic satellite images of GF2 and SPOT7 were used to do the experiments.In order to ensure the consistency of surface reflectivity, the imaging time of two panchromatic images is very close.Both images are taken in later October and located at Mount Song, Dengfeng, Henan province.There is almost no cloud in the images, but some thin mist and haze could be recognized in some local areas.Totally 60 image patches of GF2 and 40 image patches of SPOT7 were selected including building, water, farmland and remote sensing calibration field.The size of each patch is 200*200 pixels.The calibration field was established by Wuhan university and was designed to test the radiation and geometric performance of satellite image.The artificial targets placed on the ground can be used to do the MTF estimation more precisely.Table 1 shows the information of two satellite images.We introduce the fuzzy comprehensive evaluation method to synthesize these features and analyze the image quality.The framework of the proposed method can be roughly divided into following steps.

Satellite
Firstly, image normalization was done to eliminate the effects of quantification levels and radiation inconsistencies.Because the radiometric resolution of GF2 is 10 bits while SPOT7 is 12 bits, the grayscale of both images was normalized to 0-1.Secondly, the knife-edge regions were extracted from these image blocks manually.Then a feature vector was obtained containing two MTF indices and seven radiant features.In order to get the fuzzy evaluation threshold

EXPERIMENTS AND ANALYSIS
In this study, two groups of experiments were done using SPOT7 and GF2 images.Firstly, the experiment for quality assessment of different surface objects was done with GF2 image patches.
The surface objects including remote sensing calibration field, building, water and farmland.This experiment mainly analyzes the quality differences of different ground objects in details.
Secondly, in order to test the influence of different features on quality evaluation, different weight vectors were set for the experiment with GF2 and SPOT7 images.Because the GSD of two images is different, it will also be an important factor in the comparison of quality evaluation.
Before the experiment, we assigned definite scores for the five assessment grades.The quality grades set was designed as V = {1, 0.75,0.5,0.25,0},respectively corresponding to excellent ( 1 ), good ( 2 ), fair ( 3 ), poor ( 4 ) and very bad ( 5 ).Besides, the fuzzy evaluation threshold table needs to be determined in advance, it is a key component to construct the fuzzy membership function.Twenty GF2 patches and twenty SPOT7 patches were selected to make up the training set.And the training set contains the above four types of ground objects.The feature vectors were extracted from these patches.By statistical analysis of the feature vectors, the fuzzy evaluation threshold table was obtained.

A. GF2 Image Quality Comprehensive Evaluation
There are 30 GF2 image patches used in this quality comprehensive evaluation experiment, which contain 10 patches of building, 10 patches of farmland and 10 patches of water.
When extracting knife-edge regions, same patch is extracted artificially four times repeatedly.The evaluation score was obtained by putting the feature vector into the fuzzy membership function.Finally, in order to reduce random error, the mean score of four comprehensive evaluation results is obtained.Figure 4 shows the sample patches of the GF2 image.The choice of knifeedge region requires certain experience of image interpretation.
In the building area, the boundaries between building and surface or shadow are good choices.In the farmland area, the boundaries of different crops are easy to identify.And in the water area, we select the straight boundaries of water and land or dam.After extracting knife-edge regions, the MTF and radiant features are calculated.According to the above evaluation results, the analysis can be carried out by combining feature vectors of different patches, some feature vectors were shown at Table 3.First of all, for the water image patch, due to the large gray difference between water and land, the contrast is high, the image edge is relatively clear, and information entropy is also high.However, the Nyquist and MTF0.5 value is lower than the building patches.Secondly, the surface material of the building is relatively pure which makes the edge very prominent in GF2 image, therefore, the MTF features of building patch are lager.Meanwhile, high edge intensity and high contrast are also the advantages of building patch, but the PSNR is lower than other patches.At last, the image patch of farmland has low edge strength, low contrast, low entropy, only the PSNR is higher than other patches, therefore, the quality of farmland image is generally low.
If we evaluate the image quality by analyzing image features respectively like traditional ways, it is difficult to give quantitative evaluation results of different image patches.However, the fuzzy comprehensive evaluation method can obtain a clear quality score through the mapping calculation of the feature vector to the quality grade set by the membership function.

B. The Experiment of Different Feature Weights
In order to test the influence of different features on quality evaluation, different weight vectors were set for the experiment with GF2 and SPOT7 images.In this experiment, 22 GF2 image patches of different areas were selected, including one patch of calibration field, 7 patches of building, 7 patches of farmland and 7 patches of water.The grayscale distribution of SPOT7 image is the same as GF2, because both images grayscales were normalized to 0-1.The selection of different surface objects is also very similar, and some patches of the same surface objects are shown below.Firstly, the experiment with average weight was done.Figure 7 shows the result of comprehensive evaluation with average weight.First of all, the quality score of GF2 calibration field patch is one level higher than SPOT7 image.The farmland patches of SPOT7 image have higher grades, but GF2 water patches have better quality.The building patches' quality of both images are very similar.Combining the feature vectors of the table 5, we can find that among these features, GF2 image has higher GSD, entropy, average difference, edge intensity and average gradient.Therefore, while using average weight, most of the GF2 image patches' quality are higher than the SPOT7 image patches.When the weight is biased towards GF2 priority feature, the evaluation result will change to that GF2 image has more better quality.When the weight is biased towards SPOT7 priority feature, the conclusion is opposite.The reason is that GF2 image has higher resolution which make the regions separated by the boundary more homogeneous, so, GF2 image has good edge intensity and average gradient.And GF2 image has smaller gray variance, it has a uniform grayscale distribution which means higher entropy.
In the other side, although SPOT7 has lower resolution than GF2, it has a wider distribution of grayscale.The higher radiation variation results to the higher PSNR according to its computational formula.In addition, it also makes the higher contrast and bring about the sharper edges in the images, so the MTF performance in SPOT7 is better than in GF2.
The experiments prove the dependence and sensitivity of fuzzy comprehensive evaluation method on weight setting.With different weight sets, the differences in image features are revealed.Using the fuzzy comprehensive evaluation method, the quality of one image is not absolute, different evaluation results can be obtained with various feature weights.

CONCLUSION
In conclusion, we adopted the fuzzy comprehensive evaluation method on MTF and radiant features for image quality assessment of high-resolution satellite images.Generally, for the patches from the same satellite, the image quality of building and water areas is superior to the farmland, and the calibration field image has best quality.For image quality comparison of GF2 and SPOT7, different feature weights in fuzzy comprehensive evaluation method affect the evaluation results.In the case of setting edge features, entropy and spatial resolution as the main factors, the image quality of GF2 is better than SPOT7.However, while setting MTF and PSNR features as main factors, the image quality of SPOT7 is better than GF2.Our research shows that the fuzzy comprehensive evaluation method can not only evaluate the image quality comprehensively, but also can reflect the impact of diverse characteristics for different images quality assessment flexibly.
In future study, further research will be conducted on automatic extraction of image features, and testing the universality of this method for other high-resolution images.More work on extracting feature automatic is helpful to realize image quality evaluation automation.By testing the evaluation effect of this method on other high-resolution images, the applicability of this method can be improved.

Figure 1 .
Figure 1.The process of MTF calculation In addition, two image quality features based on MTF curves are calculated.The first feature is Nyquist which is the MTF value where normalized frequency is 0.5.It is often used as a parameter for the design and performance evaluation of cameras.Otherwise, Nyquist value also reflects the sharpness of the target image.The second is MTF0.5.It is the frequency value where MTF is 0.5.While the MTF0.5 is often shown in the lower frequency part in practice, it reflects the contrast on the target image.

Figure 4 .
Figure 4.The image patches of GF2 This experiment used triangle membership function which is determined by the fuzzy evaluation threshold table.And the average weight is used to analyze GF2 image quality of different ground objects.In additional, the calibration field patch's assessment result was obtained for the reference.The evaluation result is shown in the figure 5.

Figure 5 .
Figure 5.The evaluation result of different GF2 image patches The results show that the calibration field images have the highest quality scores due to the artificial sharp edges and uniform ground surface.The water images have the closest image quality to calibration field.The farmland images have the lowest scores due to the spectral mixture.And the qualities of building images are close to water images, but slightly below them.Since this research is a comprehensive image quality assessment based on local knife-edge regions, the experiment paid more attention to the detail expression of the image, and regardless of the texture features of large image ranges.The image qualities of different objects in the same image can be different.

Figure 6 .
Figure 6.Image patches of GF2 and SPOT7 at the same place Since the resolutions of GF2 and SPOT7 are different, the GSD factor needs to be taken into account.Nine features were extracted from the image patches and put into fuzzy comprehensive evaluation function.With different feature weights, the evaluation results changed.

Figure 7 .
Figure 7.The evaluation results with average weight Secondly, we improved the weights of Nyquist and PSNR gradient, and suppressed the weights of edge intensity and average gradient.The weight is shown at table 4, No.2.The result is shown in figure 8.As we can see, with the weights biased towards PSNR and MTF, almost all kinds of patches' quality of SPOT7 image is higher than GF2.

Figure 8 .
Figure 8.The results with high Nyquist and PSNR weightThirdly, contrary to the above experiment, the weights of edge intensity and average gradient were increased, and the weights of Nyquist and PSNR gradient were decreased.The new weight is shown at table 4, No.3.The result is shown in figure9.Overall, the quality of GF2 image is higher than SPOT7.But the differences of farmland and building patches' image quality are not obvious.

Figure 9 .
Figure 9.The results with high EI and AG weight The fuzzy set is calculated by the membership function and the fuzzy relationship matrix is obtained.The element   of fuzzy relationship matrix R is the degree of membership on   to   .

Table 1 .
Two kinds of satellite images used in the paper Remote sensing images are highly structured, their pixels and structures exhibit strong dependencies in space and frequency, and these dependencies carry important information about the visual scene.The MTF and radiant features can capture the information of image local structure and grayscale distribution.

Table 2 .
The fuzzy evaluation threshold tableAs we can see from table 2, since the image grayscale was normalized to 0-1, the feature values from the images with different grayscale magnitudes are comparable.

Table 3 .
The example features of GF2 image patches

Table 4 .
The weight set of experiment B

Table 5 .
The feature vectors of image patches shown above