DEVELOPMENT OF A DAYTIME CLOUD AND AEROSOL LOADINGS DETECTION ALGORITHM FOR HIMAWARI-8 SATELLITE MEASUREMENTS OVER DESERT

Satellite cloud detection is essential for the downstream cloud and aerosol retrievals. However, the detection of clouds is easily biased by aerosol loadings (smoke, dust storm, haze etc.). Currently, Cloud mask products of satellites only provide the distribution of cloud and clear-sky areas. In the environmental monitoring applications in China, the distribution of haze pollution over the North China Plain and the dust plume generated from Taklimakan Desert are poorly identified from satellites. The next generation geostationary satellite Himawari-8 is equipped with the Advanced Himawari Imager (AHI), which can provide high temporal and spatial measurements in multiple wavelengths. In this study, a cloud and aerosol loading detection algorithm over China is proposed by improving our previous Himawari-8 cloud and haze mask (HCHM) algorithm to desert regions. The HCHM algorithm classifies the AHI pixels into one of three categories: clear, cloudy or aerosol loading. It should be noted that the aerosol loading regions include haze, fog or dust layers that are easily recognized by human eyes. Based on the brightness temperature sampling results of dust storm areas in infrared bands, the tests and their thresholds in distinguishing dust from cloud and clear-sky areas are determined. Several tests [R0.46μm, BT8.6μm /BT11.2μm, BTD(11.2μm -8.6μm) and BTD(12.4μm -11.2μm)] are used to detect the dust plume from deserts. Case study results indicate that our improved algorithm can provide reasonable distribution of dust storm, clouds and clear over desert regions.


INTRODUCTION
The satellites cloud mask algorithms are generally developed to distinguish clear-sky and cloudy pixels.However, Aerosol loadings originate either from natural sources (dust, volcanic, sea salt aerosols, etc.) or from anthropogenic sources (smoke, sulfates, soot, etc.) are neither clear-sky nor cloudy regions (Poschl et al., 2010).For example, the haze pixels are usually identified as clear-conservative or cloudy-conservative in MODIS cloud mask products.Unfortunately, the haze or dust laden pixels are easily mistaken as cloudy when they overlap with cloud layers or mistaken as clear-sky over bright surface.This misidentification will bring new uncertainties to the retrieval of cloud and aerosol properties.Besides, the application of satellite data in weather forecast and environmental monitoring require the clear discrimination between haze, dust and clear.Though sensors like MODIS also provide dust or aerosol flags to indicate the distribution of some aerosol lading regions, the information are stored with subordinate level and are not easy to use (Shang et al., 2014).One of the remarkable characteristics of our algorithm is to add the third type (aerosol loading) into the cloud mask results.To make it easy to understand, the aerosol loading areas can be haze, dust or fog.The common characteristics of the aerosol loading area is that they have a relatively larger optical thickness that can be easily identified by human.To achieve this goal, we have developed the Himawari-8 Cloud and Haze Mask (HCHM) algorithm for central and eastern China, where the haze pollution is serious during winter season.In the HCHM algorithm, the pixels are classified as clear, cloudy or hazy.Unlike the traditional thresholds only stress on the tuning thresholds of different band tests, the HCHM algorithm incorporates altitude information to adjust the thresholds used in the some of the threshold tests to separate haze and cloud pixels.The principle of doing this is because haze occurs near the ground and accumulates in low-elevation plains and basins while clouds form at high altitudes.However, in the northern and western China, mineral dust is a major contributor to total aerosol loading.The dust plume has quite different spectral characteristics with haze in central and eastern China.This study aims to extend the HCHM to western China with the ability to distinguish between dust and clouds over desert regions.Several studies have discussed the detection of mineral dust using satellite observations.It is reported that the IR channels near the 8, 11 and 12 μm bands have different sensitivities to the existence of dust, therefore the brightness temperature differences (BTD) either in two or three channels is proven to be effective in the dust detection (Huang et al., 2007).For example, Ackerman (1997) demonstrated that comparing BTD between the 8 and 11 μm channels against BTD between the 11 and 12 μm channels enables to discriminate dust from the clear-sky over both oceans and lands.Generally, the atmospheric dust is grayscale, it is difficult to see dust clouds over light-coloured backgrounds or distinguish them from other cloud types.Another effective way to distinguish dust from clear sky is to create false colour image from different channels of satellite.For example, Himwari-8 AHI's measurements allows scientists to create photorealistic images of Earth by combining (12.0-10.8μm, 10.8-8.7 μm, 10.8 μm) as (red, green, and blue) channels.In these so-called dust RGB images, dust clouds show up quite clearly with pink or dark pink colour.
In this study, we investigated some of these dust sensitive tests, and determine the thresholds using dust samplings.Based on these dust detection tests, it is planned to modify the HCHM algorithm and enable the improved algorithm to detect aerosol loading over western China, especially the desert with bright surface.

Satellite datasets
The satellite data used in this study include the Himawari-8/AHI level-1 reflectance and brightness measurements and the CALIPSO level-2 daily vertical feature mask (VFM).
The AHI has 16 channels with central wavelengths ranging from 0.47 μm to 13.3 μm.As shown in Table 1, the spatial resolution is 0.5 km for Band 3; 1 km for Bands 1, 2 and 4; and 2 km for the other bands.The AHI has more than one observation modes, it can make full-disk measurements every 10 minutes, and observations of Japan and some other target areas can be acquired every 2.5 minutes.The reflectance values of visible and nearinfrared bands and the brightness temperature values of infrared bands are necessary inputs for the HCHM algorithm.Detailed information of the data can be found in http://www.data.jma.go.jp/mscweb/en/operation/type/HRIT/index.html.The vertical cloud detection information from the CALIOP vertical feature mask (VFM) products is used to locate the accurate dust storm areas in our analysis.The detection results include the vertical distribution of six feature types: cloud, aerosol, clear air, stratospheric feature, surface, and subsurface.
The resolution for the VFM products varies with the higher spatial resolution occurring at the lower altitude.The vertical and horizontal resolutions are 30 m and 333 m, respectively, at an altitude of ~8.2 km and 60 m and 1000 m, respectively, at altitudes of 8.2-20.2km (Chan and Comiso, 2011;Winker et al., 2009).

The HCHM algorithm
The

Dust storm cases
Most dust storms in China originate from one of three geographic areas: the Hexi (River West) Corridor and western Inner Mongolia Plateau, the Taklimakan Desert, and the central Inner Mongolia Plateau.Dust storm is not mainly contributed by deserts directly, instead, the atmospheric dust generated from deteriorated grasslands, Gobi, alluvial, lacustrine sediments and wadis at the outer edge of deserts (Wang et al., 2004).Six dust cases (Table 2) are collected according to history weather records.
The samplings for dust storm, clear and cloudy are conducted according to and track the dust layer by the VFM measurements.

Methods
We first investigated the dust detection tests discussed by previous studies, from with we summarized the potential tests that may be helpful for the detection of dust storm or dust plume.
It is reported that the 0.46 μm and 2.13 μm channels are useful in dust detection because the influence of dust at visible/near-IR channels are relative small/large (Qu et al., 2006).In addition, Several IR channels (8.6 μm, 10.4 μm, 11.2μm and 12.4) channels are very sensitive to the presence of dust.Yamamoto has discussed and investigated the tests using these three channels to dust over Asia for both land and ocean (http://www.aos.wisc.edu/uwaosjournal/Volume28/Yamamoto_MS.pdf).The difference of this study is that we examined the thresholds that may be useful for distinguish between dust and clouds according to CALIPSO vertical feature mask.On the other hand, as we should modify the HCHM, we will consider the test that can help detect different aerosol loadings rather than only effective in detecting dust plume or dust storm.
Figure 2 The histogram of the sampled values for over dust loading regions With the dust storm cases, we determined the typical sampling region of dust storms, and compared the spectral characteristics of both clear-sky, clouds with dust storm.After that, we determined the tests and their thresholds and combined these tests with the original HCHM algorithm, two dust storm cases over the Taklimakan desert are used to show the performance of our improved algorithm.

Dust characteristics
To determine the thresholds for dust detection, the sampling are performed over the clear-sky, dust storm and cloudy areas.The CALIOP VFM data were used to indicate where the cloud-free dust and cloudy regions exist.CALIOP can observe dust over bright surfaces and in clear sky conditions.Figure 1 (a) is the image and the corresponding VFM of a dust outbreak occurring Inner Mongolia (case 1). Figure 1 (b) corresponds to Case 2, the image and the corresponding VFM indicate the dust outbreak occurring at Henan.In Fig. 2 (a), the (11.2μm-10.4μm)/(12.4μm-8.6μm)test is examined over cloudy, dust and clear-sky regions.the results reveal that the dust pixels centred at a single peak at 0.25, the values for dust areas are smaller than clear and cloudy areas.Though a threshold of <0.4 seems reasonable to detect dust pixels, a few clear-sky pixels are mixed with the dust pixels.The test of 8.6μm/11.2μm is examined in Fig. 2 (b), in which the dust, clear and cloudy pixels peaked at 0.97, 0.99 and 1.0 respectively, a threshold of (0.98-0.99) can be used to detect most of the dust pixels; In Fig. 2(c), 11.2μm -8.6μm test is examined, the clearsky pixels cannot be found because the sampling regions are small.The dust pixels show significant difference with the PDF of cloudy pixels, most of the dust pixels have a value between 2-4.To minimize the misclassification of cloudy pixels as dust detected by R0.86/R1.6.pixels, a threshold of (0.5-4) can be used to distinguish between dust and cloudy pixels.In Fig 2 (d), the dust pixels and the clearsky pixels show very different values with cloudy pixels, however, the dust and clear-sky pixels are difficult to distinguish by the test of 12.4-11.2μm,a threshold of (-4~-3) can be used in the detection of dust pixels.These cases help us to determine the thresholds in different tests for dust detection, however, one important problem is that in desert regions, the detection of sand storm is difficult over very bright surfaces.Previous studies have indicated that some robust cloud detection test, e.g., R0.64μm, is not valid over bright surfaces, snow-covered regions or coral reefs (Remer et al., 2005).Similarly, some dust detection test may misidentify the desert as dust storm, let alone to identify the dust storm regions over the desert.The key to this issue is to use effective tests to determine the bright surface without dust plume.The ratio of the reflectance values at 0.86 μm and 1.6 μm (R0.86μm/R1.6μm) is reported effective for detecting clouds over bright desert surfaces but not clouds over dense forest (Ishida and Nakajima, 2009).In the HCHM, this test is applied to detect clear skies in highlands or mountainous areas.
In this study, we further examined the performance of these tests and found a threshold of R0.46μm>0.19 can be used to detect the bright surface without dust Besides, the test of R0.86μm /R1.6μm<0.99 is found to be effective in detecting both the desert with storm and the bright surface.Based on the above discussions, the tests and thresholds used for detecting dust is shown in the flowchart in Figure 4.

Detection of aerosol loading regions
In order to involve the dust detection in the HCHM algorithm to desert regions, we added a group of dust detection tests based on the analysis of our samplings.First, the AHI measurements prepared with all the reflectance and brightness temperature are extracted; the HCHM are applied to identify the cloudy pixels.For the remaining non-cloudy pixels, the pixels will go through the dust test.Specifically, four tests (shown in Figure 4) are applied with the same priority.The pixels pass the dust tests will be classified as dust, otherwise, the pixels are classified as clearsky.
Figure 4 The flowchat of the added tests to detect dust.

Dust detection results
To test our improved algorithm, we selected two cases over the Taklimakan Desert.The Taklamakan Desert has an area of 337,000 km2, and includes the Tarim Basin, which is 1,000 kilometres long and 400 kilometres wide.It is crossed at its northern and at its southern edge by two branches of the Silk Road as travellers sought to avoid the arid wasteland.It is the world's second largest shifting sand desert with about 85% made up of shifting sand dunes ranking 16th in size in a ranking of the world's largest deserts.Dust storms are common in the Taklimakan Desert, some reaching a height of 200 meters, cover most of the desert floor.As shown in Figure 5, The AHI captured the true-color image of sand storm on 28 and 29 in May 2018.Dust was thickest along the desert's southern margin.In Figure 4(a) and (c), nearly opaque dust not only fills the western half of the Tarim Basin in which the Taklimakan Desert sits, but even pushes past the basin's northwestern rim.The difference between these two cases is that the canter of the desert is covered by many broken clouds in Figure 4 (c).The dust's thickness may be slightly exaggerated in this image as this area has been observed near the edge of the full-disk image.By comparing the aerosol loading detection results with the natural color images, the cloudy and dust storm regions detected by HCHM algorithm was well agreed with visual interpretation.These two cases indicate that the improved HCHM algorithm is able to detect the dust storm over bright surfaces like the desert.To make the conclusion more reliable, there are still many validations need to be done, especially testing the modified algorithm in conditions without dust storm or dust plume, e.g., the hazy weather over polluted regions.On the other hand, our detected dust storm can be further validated against CALIPSO measurements.
Figure 4 The aerosol loading detection over the Taklimakan and Gobi deserts for dust storms on 28 and 29 May 2018

RESULTS
Several improvements are made to the HCHM algorithm to extend the algorithm to western China for the dust detection.We analyzed 6 dust storm cases to sample the typical spectral characteristics of dust, cloud and clear-sky.The dust storm regions are selected according to the historical weather records and CALIPSO VFM data.Our analysis indicates that the tests of R0.46, 8.6 um/11.2,11.2 um-8.6 um and 12.4 um-11.2um can identify dust plumes.These tests with our tuned thresholds are combined to make a dust detection test group.By testing the non-cloudy pixels of HCHM, the dust storm or dust plume can be identified.We applied the improved algorithm to two dust storm cases over the Taklimakan and Gobi deserts.The results showed that the dust region detected is consistent with the natural color image.This study has several limitations remaining to be done in the future work.The uncertainties of the dust detection are not quantitively examined with more cases and ground-based measurements, the dust detection over non-desert regions are not discussed in this study.
Himawari-8 cloud and haze mask (HCHM) algorithm is developed with three group of tests, i.e., R0.64, R0.86/R0.64,BT11.2, BT7.3-BT11.2,BT11.2-BT3.9, and BT11.2-BT12.3etc. The, the normalized difference snow index (NDSI).The Group 1 tests use tests of R0.64, R0.86, R2.3 and BT11.2 etc. to detect all the possible cloudy pixels.The Group 2 tests adopt R0.86/R1.64, the normalized difference vegetation index (NDVI) etc. to separate the non-cloudy pixels as clear or hazy.The Group 3 tests adopt tests of R0.86/R1.64, the normalized difference snow index (NDSI) etc. to reset the snow-or ice-covered surface as clear.The elevation data are used to assist a first-order estimate of the possibility for haze occurrence.Besides, the solar viewing angles are used to adjust the thresholds to assure the haze detection at morning or dawn hours.Unlike traditional cloud mask algorithms, the tests and thresholds in this study focus on pollution episodes over central and eastern China.Therefore, the pixels are classified into three categories: clear, cloudy and hazy.These detection results are directly used as an intermediate product to guide the retrieval of cloud and aerosol parameters in polluted conditions.The detailed flow chart and the specific thresholds for each test can be found in theShang et al. (2017).

Figure 1
Figure 1 The true color image (a) and VFM (b) for 15 March 2016; the true color image (c) and VFM (d) for4 May 2016.The CALIPSO orbit are shown in yellow line, the red box indicates the sampling region for dust storm in these two cases.

Figure 3
Figure 3 a) True-color image of 15 July 2018; b) the bright surface detected with test of R0.46; C) the cloudy regionsdetected by R0.86/R1.6.pixels, a threshold of (0.5-4) can be used to distinguish between dust and cloudy pixels.In Fig2 (d), the dust pixels and the clearsky pixels show very different values with cloudy pixels, however, the dust and clear-sky pixels are difficult to distinguish by the test of 12.4-11.2μm,a threshold of (-4~-3) can be used in the detection of dust pixels.These cases help us to determine the thresholds in different tests for dust detection, however, one important problem is that in desert regions, the detection of sand storm is difficult over very bright surfaces.Previous studies have indicated that some robust cloud detection test, e.g., R0.64μm, is not valid over bright surfaces, snow-covered regions or coral reefs(Remer et al., 2005).Similarly, some dust detection test may misidentify the desert as dust storm, let alone to identify the dust storm regions over the desert.The key to this issue is to use effective tests to determine the bright surface without dust plume.The ratio of the reflectance values at 0.86 μm and 1.6 μm (R0.86μm/R1.6μm) is reported effective for detecting clouds over bright desert surfaces but not clouds over dense forest(Ishida and Nakajima, 2009).In the

Table 1
The central wavelengths and spatial resolution of AHI

Table 2
The dust storm cases used in this study