ANALYSIS OF ACCURACY OF MODIS BRDF PRODUCT ( MCD 43 C 6 ) BASED ON MISR LAND SURFACE BRF PRODUCT-A CASE STUDY OF THE CENTRAL PART OF NORTHEAST ASIA

EOS/MODIS land surface Bi-directional Reflectance Distribution Function (BRDF) product (MCD43), with the latest version C6, is one of the most important operational BRDF products with global coverage. The core sub-product MCD43A1 stores 3 parameters of the RossThick-LiSparseR semi-empirical kernel-driven BRDF model. It is important for confident use of the product to evaluate the accuracy of bi-directional reflectance factor (BRF) predicted by MCD43A1 BRDF model (mBRF). A typical region in the central part of Northeast Asia is selected as the study area. The performance of MCD43A1 BRDF model is analyzed in various observation geometries and phenological phases, using Multi-angle Imaging SpectroRadiometer (MISR) land-surface reflectance factor product (MILS_BRF) as the reference data. In addition, MODIS products MCD12Q1 and MOD/MYD10A1 are used to evaluate the impacts of land cover types and snow covers on the model accuracy, respectively. The results show an overall excellent performance of MCD43A1 in representing the anisotropic reflectance of land surface, with root mean square error (RMSE) of 0.0262 and correlation coefficient (R) of 0.9537, for all available comparable samples of MILS_BRF and mBRF pairs. The model accuracy varies in different months, which is related to the phenological phases of the study area. The accuracy for pixels labelled as ‘snow’ by MCD43 is obviously low, with RMSE/R of 0.0903/0.8401. Ephemeral snowfall events further decrease the accuracy, with RMSE/R of 0.1001/0.7715. These results provide meaningful information to MCD43 users, especially those, whose study regions are subject to phenological cycles as well as snow cover and change.


INTRODUCTION
The MODIS Bi-directional Reflectance Distribution Function (BRDF) Product (MCD43) is of the characteristics of high temporal resolution, high spatial resolution, full optical spectral coverage as well as global imaging.It is currently one of the most important operational data sources of land surface BRDF.The MCD43 implements the RossThick-LiSparseR semiempirical BRDF (RTLSR) model and the core sub-product MCD43A1 stores 3 parameters of RTLSR model derived from 16-day high-quality land surface reflectance data (Strahler et al., 1999).The latest version of MCD43A1 is Collection 6 (C6), with a nominal spatial resolution of 500 m.MCD43A1 C6 is produced daily and uses the ninth day of the 16 days as the nominal retrieval date of the product.
A large number of validation and accuracy analysis works have been carried out on RTLSR model and MCD43A1 products (Hu et al., 1997;Huang et al., 2013;Jin et al., 2003;Shuai et al., 2008).These works show that the RTLSR model has a wide range of adaptability, MCD43A1 products have good consistency for a long period, and its accuracy can meet the requirements of many applications and researches.
Most of these efforts focus on the growing season of vegetation.In contrast, validation work on the dormant season and snow cover still need more attention.The performance of the MCD43 product was specifically studied during dormant season and snow cover (Wang, 2011;Wang et al., 2014), however, these researches mainly presented the results about albedo, instead of accuracy of BRF on different observation geometries.Independent validation of MCD43A1 C6 is just beginning.The performance of MCD43A1 C6 was evaluated using Landsat data and MODIS surface reflectances from both Terra and Aqua satellite (Che et al., 2017).Though their conclusions are convincing, their results cannot represent the performance of MCD43A1 C6 model in the directions along the sub-satellite track, particularly within principal plane and its vicinity.
The Multi-angle Imaging Spectroradiometer (MISR), also boarded on the Terra satellite, observes the Earth's surface semi-simultaneously from nine discrete view angles along the sub-satellite track.Research demonstrated that MISR agrees to the ground measurements to within ±1% in all four bands and has also shown long-term temporal stability (Czapla-Myers et al., 2014).Investigation indicates that both MODIS and MISR calibration of the reflective solar bands have been well maintained (Wu et al., 2014).Therefore, MISR land surface bidirectional reflectance factor (BRF) product (MILS_BRF) is also commonly used to test the model accuracy of MCD43 products (Chen et al., 2008;Fang et al., 2004;Lallart et al., 2008;Lucht, 1998;Lucht and Lewis, 2000;Pinty et al., 2011).However, few researches pay attention to this issue from the perspective of different phenological phases of vegetation and snow cover events.
The objective of this study is to evaluate the performance of the MCD43A1 C6 on different observation geometries based on MILS_BRF, and to preliminarily analyze phenological phase and snowfall event impacts on the accuracy of the product.

The Study Area
The study area, about 6.7  10 4 km 2 , is located in the central part of Northeast Asia (Figure 1) with temperate continental monsoon climate.The rainy season is concentrated in the summer from June to August.In winter, accumulated snow cover occurs in most parts of the study area.The vegetation of the study area has notable phenological cycles.The period from April to October is the vegetation-growing season, and from November to March is the vegetation-dormant season.The types of land use and cover of the study area nearly include all major types of the International Geosphere-Biosphere Programme (IGBP) classification system (Figure 1(b)).The north-western part of the study area is of relatively flat terrain and an agricultural area significantly affected by human activities.The central and eastern parts are dominated by forestcovered mountains.Cities, towns and villages of different sizes scatter all over the study area.
Figure 1.The study area and its land use and cover types

Data
MODIS products used in this study include BRDF/Albedo product (MCD43) (Schaaf and Wang, 2015), snow cover products (MOD10A1 and MYD10A1) (Hall and Rigss, 2016), and land cover and use product (MCD12Q1) (NASA-LP-DAAC, 2017).The spatial resolution of this products is 500 m.The MILS_BRF is used as reference data in this study (MISR-Science-Team, 2015).The spatial resolution of this product is 1.1 km.Both MODIS and MISR data cover the period of 2011-2015.

Spatio-temporal co-registration and data selection.
The MISR and MODIS data are projected onto customized accurately aligned grids of 1 km  1 km and 500 m  500 m, respectively, both in Albers equal-area conic projection, to achieve spatial matching of 1 MISR pixel with 2  2 MODIS pixels.
There are a large number of data gaps in each orbit of MISR data.In order to suppress the influence of random noise, the area proportion of the effective BRF data of each orbit of MISR product within the study area is calculated.Dates with an effective data area proportion less than 10% do not participate in the following analysis.A total of 110 days of MISR data were selected from the period of 2011-2015.The MILS_BRF data with good quality (QA = 0) participate in the comparison.
The MCD43A1 model parameters are linearly aggregated from 2  2 pixels into 1 pixel.Only the aggregated pixel with all 2  2 original pixels with best retrieval quality (QA = 0) participates the following analysis.
The MOD10A1/MYD10A1 data are optimally combined at their original spatial resolution of 500 m, i.e., the data with higher quality of the two observations per day are selected.Then the same method as MCD43A1 is used to perform data aggregation and only data with the best quality (QA=0) participate in the following analysis.The merged result is named as MCD10A1, hereinafter.

Accuracy analysis
where iso f ,

Overall Accuracy of MCD43A1 model
According to the data selection method of this paper, 878,337 groups of MCD43A1 RTLSR model parameters and corresponding 7,905,033 MILS_BRF data were found in the study area and the period of 2011-2015.These data cover forests, grass, crop, urban pixels and their annual phenological cycles.For all the available comparable data, the MCD43A1 product shows overall excellent performance, with / rms eR = 0.0262/0.9537, in representing the anisotropic reflectance of land surface (Figure 2(d)).For all the available comparable samples, the accuracy of backward scattering (RAZ > 90°, using the RAZ definition of MILS_BRF, hereinafter) is better than that of forward scattering (RAZ < 90°), and the same is true in the nPP and XP direction subsets.In contrast, the forward/backward scattering accuracies of the nCP direction subset are similar, with / rms eR = 0.0154/0.9689and 0.0233/0.9719,respectively.(Figure 3  For common pixel of each month of the growing season (from April to October), the accuracy of April is still the highest ( / rms eR = 0.0134/0.9333)and June has the lowest accuracy ( / rms eR = 0.0298/0.8261).

The Impact of Phenological Phases on the Accuracy of MCD43A1 Model
The variation of accuracy of the model is related to the phenological phase of the study area.In general, the error of MCD43A1 model, accompanying phenological phase change of the study area, experiences a process of "continuous rise, decline, stability, and rapid rise".(Figure 4).

The accuracy of MCD43A1 model for snow pixel.
During the 16-day window for the inversion of MCD43A1 BRDF model parameters, if on the nominal production date (day 9) the MCD10A1 pixel are covered by snow, other days with snow covered pixels will be selected together for retrieval of model parameters, and vice versa (Wang et al., 2013).There are only 64 groups samples with MCD43A1 parameters retrieved as high quality snow pixel.The accuracy of MCD43A1 of these samples, with / rms eR = 0.0903/0.8401, is obviously lower than that of the same pixels when free of snow cover, with / rms eR = 0.0242/0.9505.The BRF prediction error of MCD43A1 model with parameters retrieved as snow pixel are mainly concentrated in +60.0° and +70.50° (MISR camera Ca and Da). Figure 5 shows the model predicting BRF (magenta) and MILS_BRF (cyan) at each observation zenith angle, where the bold lines represent the average value, and left panel for snow retrieval and right panel for snow-free retrieval.Ephemeral snow cover event decrease the model accuracy significantly ( / rms eR = 0.1001/0.7715).Under ephemeral snow cover event, the model accuracy for all 4 major land cover types of the study area reduces obviously.Figure 6 shows the details for (a) forest, (b) grass/cropland, (c) mixed, and (d) all land cover types.

DISCUSSION
MILS_BRF is of significant importance in testing the performance of MCD43A1 BRDF model.MILS_BRF and MCD43A1 products are synchronous, have comparable spatial resolution, and possess 4 similar spectral channels, whilst on the other hand, the MILS_BRF has wider range of sun-view relative azimuth angle and view zenith angle, and, particularly, the observation capability in the directions along orbit and near principal plane.This provides the possibility of testing extrapolating accuracy of MCD43A1 BRDF model in these crucial observation geometries.In contrast, results of analysis based on Aqua/MODIS and Landsat data (Che et al., 2017) can only represent the performance of MCD43A1 C6 BRDF model in the directions of cross orbit or cross principal plane.From this perspective, results of this paper complement a larger part of the whole picture of testing MCD43A1 C6 BRDF model in hemisphere space of observation.
Admittedly, the MILS_BRF itself is a satellite retrieval product and there exists uncertainty compared to ground-based BRF observations.However, research (Czapla-Myers et al., 2014) demonstrated that MISR agrees to the ground measurements to within ±1% in all four bands and has also shown long-term temporal stability.Also, investigation (Wu et al., 2014) indicates that both MODIS and MISR calibration of the reflective solar bands have been well maintained.These results provide confidence for MILS_BRF to be used as comparison reference.
MISR is not uniform in the sampling of the whole hemisphere, and there exists significant spatial and temporal variation of MILS_BRF of different observational geometry.Therefore, complete evaluation of MCD43A1 in hemisphere space of observation requires data from more view angles.

CONCLUSIONS
In this study, a typical region in the central part of Northeast Asia is selected as the study area and the performance of MCD43A1 C6 BRDF model is analysed in various observation geometries and phenological phases, using the Multi-angle Imaging SpectroRadiometer (MISR) land-surface reflectance factor (BRF) product (MILS_BRF) as the reference data.In addition, the impacts of land cover types and snow covers on the model accuracy are evaluated using MODIS land cover type product and snow cover products.
The results show the overall excellent performance of MCD43A1 C6 product to represent the anisotropic reflectance of land surface with root mean square error (RMSE) of 0.0262 and correlation coefficient (R) of 0.9537 for all available comparable samples of MILS_BRF and BRF predicted by MCD43A1 model.The model accuracy varies in different months, which is related to phenological phases of the study area.The accuracy of MCD43A1 model of pixels labelled as 'snow' by MCD43 is obviously low with RMSE/R of 0.0903/0.8401.Ephemeral snow fall events further decrease the accuracy of MCD43A1 model with RMSE/R of 0.1001/0.7715,though the MCD43A1 model parameters are labelled as 'best quality, full inversion' and 'snow free'.These results provide meaningful information to MCD43 users, especially those, whose study regions are subject to phenological cycles as well as snow cover and change.

Figure 2 .
Figure 2. Overall accuracy of MCD43A1 model.The 4 sub-figures of Figure 2 represent model accuracy within directions of nCP, nPP, XP, and all RAAs, respectively.The horizontal axis of each sub-figure represents the value of MILS_BRF, and the vertical axis represents the value of BRF predicted by MCD43A1.The inner (outer) frame lines represent the range of values [0, 1] ([-0.1, 1.1]).The gray slash is 1:1 line, and the color line is the regression line.R, E, and b are the correlation coefficient, RMSE, and bias of model respectively, and n is the number of samples.Moreover, the MCD43A1 model shows high accuracies for all the 4 major land cover types of the study area, i.e. forest, grass/crop land, urban, and the mixed, with 0.0265 rms e  (the mixed land cover type with the largest error), 0.9340 R  (urban with the lowest correlation coefficient).
At different RAAs, the MCD43A1 model shows highest accuracy in nCP directions with / rms eR = 0.0190/0.9761(Figure 2(a)) and lowest accuracy in nPP directions with / rms eR = 0.0303/0.9393 (Figure 2(b)).The accuracy in XP directions is between that of nCP and nPP (Figure 2(c)).

Figure 3 .
Figure 3.The accuracy of MCD43A1 model on different (a) RAAs and (b) VZAs.

Figure 4 .
Figure 4.The impact of phenological phases on the accuracy of MCD43A1 model.In row 2 of Figure 4, lines with + and ◊ represent

Figure 5 .Figure 6 .
Figure 5.Comparison of accuracy of MCD43A1 model for snow pixels with that for the same pixels when free of snow