THIN CLOUD DETECTION METHOD BY LINEAR COMBINATION MODEL OF CLOUD IMAGE

The existing cloud detection methods in photogrammetry often extract the image features from remote sensing images directly, and then use them to classify images into cloud or other things. But when the cloud is thin and small, these methods will be inaccurate. In this paper, a linear combination model of cloud images is proposed, by using this model, the underlying surface information of remote sensing images can be removed. So the cloud detection result can become more accurate. Firstly, the automatic cloud detection program in this paper uses the linear combination model to split the cloud information and surface information in the transparent cloud images, then uses different image features to recognize the cloud parts. In consideration of the computational efficiency, AdaBoost Classifier was introduced to combine the different features to establish a cloud classifier. AdaBoost Classifier can select the most effective features from many normal features, so the calculation time is largely reduced. Finally, we selected a cloud detection method based on tree structure and a multiple feature detection method using SVM classifier to compare with the proposed method, the experimental data shows that the proposed cloud detection program in this paper has high accuracy and fast calculation speed.  Corresponding author. Li Jiatian ljtwcx@163.com


INTRODUCTION
In the process of remote sensing image preprocessing, cloud cover is an important factor which lowers the accuracy and utilization rate of remote sensing images.Therefore, suitable cloud detection method and processing technology have always been the difficulty of photogrammetry.Especially with the diversification and universality of photogrammetry platform, the approach of acquiring remote sensing images is becoming easier for ordinary individual and corporation.For these vast amounts of remote sensing data, the automatic identification of cloud images has become even more important.Existing cloud detection methods can be mainly divided into two types: multi-spectral threshold cloud detection method and pattern classification-based cloud detection method.
The multi-spectral threshold detection method is more intuitive and easy to calculate, and is suitable for real-time cloud detection in remote sensing images.So it is widely applied in multi-spectral space photography measurements.Some papers used the multi-spectral images of MODIS and LANDSAT satellite to analyze the reflective properties of clouds and other objects in multiple channels, and find the best classification threshold (Choi et al., 2004a;Li et al., 2005a;Feng et al., 2015a).However, these methods are greatly affected by environmental changes, and have a limited scope of application due to the limitation of photographic equipment.It was proposed to use image features in combination with different classification methods to recognize the clouds.For example, Cao et al. (2007a) have used the fractal and gray level co-occurrence texture features to reflect image's gray property and space relationship between cloud and the earth surface to classify the images.Yan et al. (2010a) extracted various image features and used a minimum distance classification method to identify the cloud images, through examination, this method was verified to be feasible.Tao et al. (2011a) has used spectral threshold, texture, ASM, entropy and other features to gradually detect cloud images.Cloud detection method based on feature classification can be applied to different photogrammetry platforms, so it can be widely used in many circumstances.
In the progress of multi-period measurement and single-period measurement, because of the effects of the time transformation, angle of the camera and the airflow, the area covered by clouds in the remote sensing images can usually find matching underlying surface images.Under the premise of getting the matching images, we introduce a linear combination model of cloud images to extract the pure cloud images, and then recognize the cloud part by different image features.This method can effectively weaken the interference of the underlying surface information in the detection of cloud images.
Various images features such as grayscale and texture are used here, each feature has different effect in the detection progress.
To select the optimum combination of these features, AdaBoost Classifier is introduced to train a classifier and recognize the cloud parts.Finally, an appropriate clouds classifier is generated and applied to different circumstances.The experimental results show that the cloud detection method in this paper can detect the thin clouds in the remote sensing images, and can be used in automatic cloud detection well.

Matching of the cloud images
This paper uses the grayscale to detect the cloud images.For a grayscale digital image, the grayscale value of each pixel is the most basic and intuitive feature of the image (Chen et al., 2009a).Because the cloud has higher reflectivity, the gray value of cloud is much greater than the value of objects on the earth.This is the basic theory of spectrum threshold method, which was applied to detect the cloud at the very beginning.In this paper, the mean and variance of image grayscale were chosen to recognize the cloud images at the first step.In the process of calculation, each image was divided into N×N sub-images, let f(m, n)be the grayscale value of the row m and column n in each block, then the meanf1 and variancef2 can be calculated as follow: (1) Comparing the calculated value with the empirical threshold can exclude the objects with low reflectivity.The final results of the first step can be divided into clouds and other objects.But due to the inaccuracy of grayscale threshold, many ground objects which have the same reflectivity will be misclassified.In order to make sure that all the cloud images are selected in the first step, the empirical threshold was set in a more loose range.In multi-time and single-time measurement, there are many uncovered images which contain the same scope as the cloud images.Figure 1 shows three adjacent aerial photos during the same period of a measurement, the overlapping rate of the image is more than 75%, and the underlying surface information in the overlapping areas is basically the same.Due to the transformation of the camera angle and airflow, the bottom of the middle photo is covered by clouds, while the adjacent photo is integral.This situation often occurs in photogrammetric images.We can usually use the overlap rate to find the same area in adjacent photos.

Figure 2. Different size of clouds in photogrammetry
However, for a wide range of cloud areas, there may be multiple adjacent images which are covered, then the cloud image can't find it's matching dnderlying surface images.As shown in figure 2, the cloud is thin and small in the left picture, image a is covered but the adjacent image a0 is integral, so the matching image can be found, the cloud is thick in the right picture, both two images a0 and a are covered by the cloud, so the surface images of the central area of the cloud can't be found.But it is apperent that when the cloud is thick it is always opaque, so there is no need to deal with these images by linear combination model.So, in the first step we directly defined the cloud images which can't find its matching images as thick cloud images.After the first step, all the cloud images can be divided into two types: thin cloud images with corresponding underlying surface images and thick cloud images without corresponding underlying surface images.

Linear combination model
Because clouds are made of steams which are usually transparent, so the images of cloud contain not only cloud information but also information of the objects on the earth.Traditional cloud detection methods usually directly extract the image features from the mixed images of clouds and surface objects, these methods neglect the underlying surface information which may influence the accuracy of the cloud detection.This phenomenon is just as same as the smoke in (Tian Het al., 2014a).So in this paper we introduced the linear combination model of cloud images to deal with the cloud images.
When there are clouds in the image, according to the atmospheric scattering theory and the incident light attenuation theory, the reflected light of the objects on the earth passes through the clouds and reaches the camera, finally it will decay.And the clouds will also reflect the light to the camera, which means that the light received by camera is composed by the reflected lights of clouds and surface objects.Therefore, the cloud images are mixture of underlying surface information and cloud components.The composition of a mixed image can be represented by the formula as follows: In these formulas m represents the number of iteration.At first we set an appropriate initial value, after several iterations the optimum value can be obtained.
Figure 3 shows two pictures which can explain the effect of using this linear combination model.The left image is the marked cloud image which was detected by the first step, and the right image is the pure cloud image obtained by linear combination model.As we can see, the edge of the cloud in the left picture is thin so the underlying surface information can be seen in the left picture.In the traditional cloud detection method these surface information may influence the detection accuracy.By using the linear combination model we get the right picture, the underlying surface information is removed, so in the next detection progress we can better detect the cloud part.

Figure 3. Image comparison using linear model
The linear combination model makes full use of the adjacent underlying surface images that we can get from the remote sensing images.By removing the interference of the surface information the detection accuracy of thin clouds is largely improved in the following algorithms.

Image features
For all of the cloud parts, we finally use the image features to recognize the clouds.At present, the most common cloud detection algorithms based on image features mainly rely on the features such as grayscale, texture and frequency (Hou et al., 2014a).Different features represent different properties of the image, and for the cloud detection accuracy every feature has a unique effect.This paper selected the grayscale and texture features of the image to recognize the cloud images.
Commonly used grayscale features are mean, variance, difference, entropy and gray histogram.Using grayscale features can roughly separate clouds from the objects on the earth, but for some objects have the same reflectivity as cloud, such as snow and roof etc., this method will cause wrong detection.So the method using grayscale features is always used to detect the approximate areas before other methods are used.
The method which uses texture feature of the image is very widely used and has high accuracy.In each image, although the shape of the cloud is changing all the time, the textures of the cloud and object on the earth are very different from each other.Therefore, the method using the texture can detect the clouds well.An effective method to extract the texture is using the gray level co-occurrence matrix.The gray level co-occurrence matrix is a matrix representation calculated from the spatial correlation of two pixel gray levels in an area.From this matrix the ASM, CON, IDM, ENT and COR can be calculated.These parameters can represent the correlation of the adjacent pixels.So many papers used the gray level co-occurrence matrix to get the texture feature.Figure 4 shows some samples during the training process, the cloud images are positive samples and other images are negative samples.The features of these samples were calculated and input into the classifier.

AdaBoost classifier
Because this paper selected variety of image features to detect the cloud, taking into account of the computational efficiency and detection accuracy of the cloud detection program in this paper, the AdaBoost Classifier was introduced to deal with these features.
The AdaBoost Classifier is a very effective one in many classification method at now.Wang et al. (2013a) has used this classifier in the cloud detection and get a good result.The result of the experiment in his paper shows that this method improved the accuracy and reduced the calculation time when comparing this classifier with the traditional SODATA clustering method.AdaBoost algorithm is an iterative algorithm, its core idea is to train different classifiers which are called weak classifiers on the same training set through different features, and then combine these weak classifiers according to a certain weight to form a stronger final classifier.
In the detection program of smoke images, we first extract the image features of the video frame according to the method described above, and then input these image feature to the training program.During the training program each image feature can generate a weak classifier.After all the weak classifiers satisfy the classification accuracy, these classifiers are combined according to certain weight values.As describe above, we can get many features to detect the clouds.But too many features prone to reduce the computational efficiency in traditional classifiers.By using the AdaBoost algorithm, we can select several optimum features according to the required accuracy.Figure 5 is the AdaBoost structure diagram, in this picture the wn (M) represents the weight, Gm(x) represents the weak classifier, and G(x) is the final classifier.The final classifier is composed of each weak classifier based on different weights.
Figure 5. AdaBoost structure diagram In the training process of weak classifier, each sample is giving the same weight at the beginning, so we can get the initial sample set of weights as follow: the samples will be classified according to the sample weights, the basic weak classifier is: After the basic classifier is trained, we can calculate the error rate of the classifier Gm(x) on the training dataset: According to the error rate, calculate the weight of the weak classifier: Weak classifier training is a gradual iterative process.In the subsequent process, the weight distribution of the sample needs to be updated using the last calculated value.The calculation formula of the update process is as follows: According to the above steps, we can get several weak classifiers which can meet the accuracy requirements after several iterative calculations from feature values.Combine these classifiers linearly to get the final classifier: According to the methods proposed in this paper, design the automatic cloud detection program in remote sensing images.Figure 6 is the flowchart of cloud detection in this paper.

EXPERIMENT ANALYSIS
According to the method and flowchart proposed upside, we selected VS2013 compiler and OpenCV graphics process library to build an automatic cloud detection program for remote sensing images.About 1000 images in Jianshui, Yunnan Province are used to test the program, and there are 500 images are covered by cloud.The size of the image is about 2400× 1600 pixels.Divided all samples into 5 subsets, of which 4 were used to train the classifier and the other subset was used to test the classification effect.In the calculation process each images was divided into 64×64 sub-images, calculated the value of the grayscale and texture of each sub-image and then inputted them into the classifier to classify the images.
In order to test the performance of the proposed method, this paper selects two cloud detection methods to compare with the program in this paper, these two methods are: a cloud detection structure which is a tree-like discriminate structure composed of a series of characteristic thresholds (Shan et al., 2009a), and a multiple feature detection method using SVM classifier.SVM is a better one among many classifiers and is widely used in the classification of remote sensing images (Fu et al., 2010a;Zhao Min et al., 2012a).Table 1 shows the detection accuracies and computational efficiencies of these three methods.As we can see from the table, because this paper uses the linear combination model, the interference of the ground information in the mixed image is removed, so the detection accuracy is improved and the missing rate of thin cloud is lower when compared with the tree structure detection method that directly uses the image features.The classification process mainly uses the optimum threshold trained by the AdaBoost classifier, the image data to be tested does not need too much processing, so the computation is faster than using the SVM classifier.

G(x)
Figure 7 shows the results of different methods, the area containing the clouds is marked with red square.Because the clouds in the edge are thin and transparent, it is hard for the existing method to detect these clouds.Shown in the left image is the result of the tree structure method, and the edges of the cloud are unmarked in the left picture.The right picture is the result of the method in this paper, almost all the clouds are marked, which shows that the method using the linear combination model can better recognize the thin cloud.

Figure 1 .
Figure 1.Adjoin remote sensing images =  + (1 − ) +  (3) In this formula I represents the mixed image, C represents the clouds ingredient, and B represents the underlying objects.Parameterα is the weight of clouds in the mixed image, and ζ is the model noise.In the actual situation, I and B are usually known.C and α can be calculated through minimizing the noise parameter ζ.In a N×N size sub-image, f∈I, c∈C, b∈B, then the formula can be written as follow: N+1 variables, but in a sub-image there are only N equations.In order to make the solution unique, we need to constrain c or α to get the result.In different circumstance, the background b varies greatly with the underlying surface information, but the clouds tend to have relatively consistent visual characteristics, so we can considerate to restrict the cloud ingredient c to solve the equation.This paper introduced a local smoothing model to restrict the cloud ingredient.In a small cloud sub-image, each pixel generally has similar image features such as brightness and color as neighboring pixels, which means that the clouds in the sub-image tend to be smooth.By using the local smoothing model, we can get the equation as follow:  , ‖ −  − (1 − )‖ 2 2 +  ∑ ∑ (  −   ) equation, we get the value of c and α as follow:   = (  2  + ) −1   ( −  +   )   )  (−) (−  )  (  −)

Figure 4 .
Figure 4. Positive and negative samples

Figure 7 .
Figure 7.The images of different methods results