ANALYSIS OF QUALITY REVIEW OF LAND COVER CLASSIFICATION PRODUCT IN GEOGRAPHIC NATIONAL CONDITIONS MONITORING

To the land cover classification product of geographic national conditions monitoring, how to scientifically and objectively inspect and evaluate the results is a problem that if quality review can play a key role in quality control. Based on the characteristics of the land cover classification product, it was carried out a quality inspection for important quality elements and key contents on the basis of quality acceptance to ensure the consistency and compliance of the national results. This paper put forward the content and method of review. The results over the years since the review showed that the classification accuracy was not only a difficulty but also a short board in quality control. Through summarizing the main problems of the land cover classification product found in the review, the causes were analyzed, and suggestions were put forward for production. *Wenjuan Mao, National Quality Inspection and Testing Center for Surveying and Mapping Products, Beijing, China 67516521@qq.com


INTRODUCTION
In recent years, many countries and organizations have carried out projects or projects similar to the monitoring of geographical conditions to provide data support for major national planning and action plans. With the expansion of application fields and the improvement of application levels, the monitoring of geographic and national conditions has entered a stage of normalized monitoring after the completion of the first national geographic and national census in China. Geographical monitoring of national conditions forms monitoring data and research reports that reflect the spatial distribution and development changes of elements such as the ecological environment, social economy, etc. (Xu, 2012) It is an effective way to accurately grasp the national conditions and national strength from the perspective of geographic space (Shi et al., 2013; Ministry of Natural Resources of the People's Republic of China, 2019). Among them, as collecting land cover information is the primary task and the basis of geographical and national conditions monitoring, land cover data can truly reflect the types and attributes of land surface materials and become an important method for describing geographical and national conditions information (Cheng et al., 2013).
Since 2016, the annual monitoring of geographic conditions has been carried out for four consecutive years. Its reliability and quality directly affect the correctness of decision-making and the lack of quality-assured geographic information will not be able to provide effective support for scientific decision-making (Shi et al., 2012). The authenticity and reliability of the results are particularly important. As an important content, the quality control of the monitoring results has been continuously followed up with the progress of the project. According to the characteristics of project organization and implementation, a comprehensive quality control system from process quality control to result acceptance and quality review has been constructed and quality supervision and management work runs through the whole process of production.
Through the comprehensive inspection and acceptance of the monitoring results, the quality of the results can be scientifically and objectively evaluated to ensure the comprehensiveness, authenticity and accuracy of the monitoring results. In order to ensure that the inspection and acceptance work of each province is standardized and effective and to further ensure the authenticity, accuracy, reliability and consistency of monitoring results across the country, the quality of the results will be reviewed from 2017.
As the last link in the quality control of geographical and national conditions monitoring, the quality review of results is the last check on the quality of the results. It focuses on the important contents, key indicators and weak links in monitoring and monitoring. A unified scale is used to measure the acceptance scale of each province nationwide by reviewing the key inspection of key monitoring results after passing the acceptance of each province. Acceptance is a comprehensive inspection of all the elements of all the results, while the review of results is mainly aimed at important results and important quality elements. This paper analyzes the quality review situation in the past three years and discusses the effective way to improve the quality of the results.

DATA AND STUDY AREAS
Important results are the land cover classification results, the geographical national conditions elements results and the results of production metadata. Important quality elements are some of the important inspection items of the three types of monitoring results. Among them, the land cover classification results are the geographic information content with seamless and full coverage. Because of its novel form and wide application, it has always been the focus of quality control in the monitoring results of national conditions and is also an important guarantee to ensure the effective application of the results (Bi et al., 2014;Zhang et al., 2015;Chen et al., 2018). This paper analyzes the quality review of 26 provinces across the country for three consecutive years from 2017 to 2019 and explores effective ways to improve the quality of results.

Review content of land cover classification results
Review is an inspection based on acceptance. It aims at the key content in the application of the results and focuses on the quality problems that are prone to occur and have a great impact in the production of the results. Through the review, it is first necessary to confirm whether the quality of the results meets the acceptance requirements and secondly to accurately reflect the existing problems and their severity. In the case of fully considering the benefits of input and output, it is very important to determine the content and methods of the review. Ma et al., 2018) According to the requirements of GB/T 39613-2020(Quality Inspection and Acceptance of Geographical National Conditions Monitoring Results), the important quality elements refer to the collection accuracy and classification accuracy for the classification results of land cover (Standardization Administration of the P.R.C., 2020). See Table 1 for the review content of land cover classification product. It can be seen from the table that controlling classification accuracy and collection accuracy is the focus of quality review. The classification accuracy includes the correctness of the patch classification and the completeness of the change identification, which is the top priority of the quality of the results. The results of the classification accuracy directly affect the comprehensiveness, authenticity and accuracy of the annual statistical results. Therefore, the quality element of classification accuracy can be called the core of quality review of land cover classification results.

Important
Judging from the inspection content, the collection accuracy focuses on checking the first-class and second-class classification of patch fit and edge problems. GB/T 39613-2020 requires that the collection accuracy is directly judged to be unqualified if the position of the data boundary is modified with the image or the overall image spot in the map is over the limit of the orthophoto without deducting the overlap difference. So for the areas with large changes during the inspection, it is necessary to perform an overlay analysis of the background and monitoring images to determine whether there is any false change in the update of the edge of the patch.
Classification accuracy aims at checking whether there are errors or omissions in the classification of patches. In GB/T 39613-2020, it is required that classification accuracy is directly judged to be unqualified if a single area exceeds 20,000 square meters patch is erroneously updated that has not changed during the monitoring period. Therefore, the classification accuracy focuses on the correctness of the classification of patches with a changed area of more than 20,000 square meters, errors and omissions in cross-level class updates, annual key monitoring contents and new additions. Normal monitoring will arrange new monitoring priorities every year, such as it is required to focus on the accuracy of changes in land types of paddy fields, arable land, road surfaces and hardened surfaces in 2019.

Review method of land cover classification results
(1) Sampling method Due to the characteristics of review, the random sampling method for sample selection can achieve the purpose of review.
In the review of the land cover classification results, the sampling unit which is consistent with the acceptance is carried out with the county-level survey area. Since the review is to confirm and verify the results that have passed the acceptance inspection in each province, the number of review samples is determined to be no less than 20% of the number of provinciallevel acceptance samples. In order to avoid too few review samples that cannot truly reflect the quality status each provincial-level task undertaking unit shall draw at least 2 county-level review samples in principle. At the same time it is required that the sampling should take into account the acceptance samples and non-samples in a ratio of 1:1 to ensure the comprehensiveness of the review work and the validity of the review conclusions. Considering the application requirements of the monitoring results, the review stage mainly focuses on the correctness of change identification. Therefore, the map of the area with large changes (around towns or major engineering projects) is no less than 10% of the area in the sample is selected for key inspection on the basis of sample sampling in the county-level survey area to ensure the maximum reflection of the true and complete coverage of changes.
Due to the large impact caused by the wrong classification of patches with an area of more than 20,000 square meters, more than 2 additional county-level survey areas are selected to conduct a special inspection of large patch classification errors and omissions on the basis of the detailed inspection of the samples.
(2) Inspection method The land cover classification results are mainly checked by comparing with the image data used in production to check whether the collection accuracy of the patches exceeds the limit. It is determined whether there are errors or omissions in the classification accuracy through reference comparison with image data, field inspection data or photos in production and remote sensing image interpretation sample results., The errors and omissions of the results are checked through the method of field inspection for the patterns that cannot be judged by the internal industry.
In order to play an effective role in the review work, it is necessary to highlight the problem-oriented principle to meet the pertinence of the review. Before reviewing the results of each province, get a detailed understanding of the main problems found in the acceptance of each province's results especially the generality and tendencies. In the review of results, it is necessary to focus on the main quality problems in the quality control of the early process.
(3) Evaluation method In order to ensure the effective connection between review and acceptance, the quality evaluation index is implemented in accordance with GB/T 39613-2020 in the review evaluation. The sample quality element score takes the minimum value of the sample map in the county-level survey area and the review conclusion is based on the quality element evaluation results. If and only if all the sample quality element scores meet the qualified conditions and no large-area patch classification errors or omissions outside the samples are found, the review of land cover classification results is judged as pass. If any of the quality elements is unqualified or any large wrong classification pattern is found outside the samples, the review conclusion is not passed.

Overall quality
Through statistical analysis of the review results of 26 provinces across the country for three consecutive years from 2017 to 2019, the pass rate of one-time review for three important types of results is shown in Table 2. The results show that the onetime review pass rate of land cover classification results is much lower than that of the other two types of results. Further analysis found that the reasons for the failure of the review of land cover classification results were that the sample classification accuracy was not up to standard or any more than 20,000 square meters pattern classification errors and omissions were found.

Results of Production Metadata 2017
Below 80% More than 90% 100% 2018 Below 70% More than 90% 100% 2019 Below 70% More than 90% 100% Table 2. Pass rate of quality review The classification accuracy is evaluated by counting the proportion of bad areas. The limit of qualified indicators are the bad area for 0.1% of the first-level error and 0.4% of the second-level and third-level error. For the nationwide 1:10,000 sub-scale review samples, the average area of the 1: 10,000 sub-scale results in different regions is about 26 km².In the review, it can be intuitively understood that the first-level bad area cannot exceed 26,000m² and the second-level and third-level bad area cannot exceed 104,000m². In 2019 the superior management department adjusted requirements of review approval: the quality element score of the three types of key results was not less than 70 points and no classification errors and omissions outside the sample was found. That is to say, for a 26km² 1:10000 framing result, the first-level bad area cannot exceed 19,500m² and the second-level and third-level bad area cannot exceed 78,000m². This limit is a key technical indicator in monitoring production and quality inspection to ensure the consistency of the quality of results across the country.
The quality element scores of land cover classification result samples in the 142 county-level survey areas that have passed the three-year review are counted as shown in Figure 1and   More than half of the samples collection accuracy score were more than 90 points, while the classification accuracy scores were concentrated in 70-90 points among the samples that passed the review.
The average score and the average minimum score of collection accuracy and classification accuracy in 3 years are counted as shown in Figure 3 and Figure 4.

Major quality issues
By summarizing and sorting out the quality problems of the three-year review, the problems of land cover classification results are mainly reflected in the following aspects.
(1) Classification accuracy The problem of classification accuracy focuses on two aspects: misclassification and missing classification.
Misclassification is an obvious classification error. The first type mostly occurs in the mixed classification of ground objects such as forest land and grassland in the natural state or artificial state. The second is the update of the patch without basis and most of them are more than 20,000 square meters. The third is the assignment error of the patch classification code caused by the incorrect operation of the merged patch. The fourth is that the patch classification is inconsistent with the field inspection results.
Missing classification refers to the obvious changes in the ground features that are not updated.
(2) Collection accuracy From the average score and the average minimum score, the collection accuracy score was generally higher than the classification accuracy.
GB/T 39613-2020 requires that serious fit overruns should be classified as classification accuracy errors. It is found that the fit over-limit error is classified as a classification accuracy error in the review which affects the classification accuracy score. This kind of situation also increases the difficulty of controlling the classification accuracy to a certain extent.
By analyzing the three-year review results, it can be seen that the land cover classification results are more likely to have problems affecting the review conclusions than the other two types of important results. The classification accuracy is not only a difficulty in controlling the quality of the land cover classification results, but also a shortcoming.
The problem of collection accuracy mainly focuses on that the collection of image spots is too large; the second is that the overlap between spots and images exceed the limit.

Cause Analysis
The geographical monitoring of national conditions is a continuous project carried out by the provinces on the basis of the first national geographic and national census. The technical standards are unified and the organization and implementation are strict. The application prospects of the results are generally optimistic.
However, due to the differences in the investment situation of the provinces, the deficiencies in organizational management and technical processing are directly reflected in the quality of the results and the review results still need to be improved.
At the same time, considering the particularity of the results, the improvement and refinement of technical design is also the key to quality improvement. Especially if the provincial design for the characteristics of each province is not well targeted, it will directly cause problems in the quality of the results. Among them, the problem of easy-to-mix patch classification in content and indicators has always been the crux of the quality of results and it needs to be actively explored.
Then the technical ability of the operators is also the key point that affects the quality of the results. The lack of careful and inplace training has resulted in the operators not fully grasping the key contents of monitoring. It has caused that the key issues are not clear and the new annual requirements are not understood. It has led to repeated problems of the quality of the results by carrying out the geographical and national conditions monitoring operation by means of census or other monitoring projects especially for those who are new to the monitoring operation.

Suggestions for improving the quality of the results
The annual review work ensures that the process quality control and acceptance work play a practical role. While discovering problems and guiding the revision of the results, we understand the provincial acceptance conditions and master the quality level of the national results. It is beneficial to strictly control the quality of the final results by comprehensively understanding, accurately diagnosing the implementation of the project and prescribing the right medicine and provides opinions and suggestions for monitoring production optimization and continuous quality improvement.
(1) Aim at the application of results and do a good job in quality control.
The application of results is an important indicator to reflect the value of the results. With the application as the demand and the continuous improvement of the quality as the goal, the quality control method suitable for the region and the department can be continuously sought to ensure solid, detailed and good quality.
(2) Optimize technical design and enhance the pertinence of design.
Technical design needs to continue to conduct in-depth research and analysis of difficulties and blind spots in production and continue to improve around quality issues. It is crucial for each province to formulate a practical and feasible plan for its own situation based on the overall requirements of the project.
(3) Strengthen the training effect and ensure the ability of personnel.
Provinces carry out targeted training in combination with annual monitoring priorities and new requirements. In order to eliminate hidden dangers in the quality of later operations in the training process, it is necessary to ensure that the training is in place so that operators truly master the technical requirements.
At present, the review work is fully integrated with the results of the annual process quality control. During the process inspection, it is found that changes in the production team or large-scale production personnel will lead to the continuous improvement of the quality of the results and there is a risk of continuous improvement in the quality of the results. It is necessary to focus on these problems in sample selection of reviewing to ensure the quality management system. In order to serve government decision-making better, more and more attention needs to be paid to application requirements with the increasing application of projects. Therefore, as the last step of national quality control, the selection of samples in the review needs to be more targeted. A more targeted approach is needed for areas of concern to hotspots or areas where there is doubt about the results of previous year's statistical analysis

CONCLUSION
The above results fully confirm the particularity of the land cover classification results which are highly consistent with the surface morphology. It changes the symbolic features of the points, lines and surfaces of basic geographic information elements, which is undoubtedly a challenge for operators who are accustomed to basic surveying and mapping production. It is necessary to be soberly aware that the land cover classification results are the results of patches based on the classification of content and indicators. The aim is to locate the locations, attributes and areas of patches whether in production methods or technical requirements, which is the important content to ensure the quality of the results.
Monitoring geographical conditions has become an urgent need to promote scientific management decision-making. The objectivity and accuracy of result data are the lifeblood of the project and ensuring the quality of results is fundamental. The content and method in the quality review of land cover classification results mainly discussed in this paper have been effectively applied in the quality review of the geographical and national conditions monitoring results.
The core difficulty in reviewing land cover classification results is to grasp the quality elements of classification accuracy. References are provided for monitor production, process quality control and achievement acceptance for the next year by quality quantification and problem analysis in annual reviewing. In order to unify the quality level of the national results, effective measures and methods will be adopted in the review to guide the continuous improvement of the quality of the results nationwide to ensure that the quality of the final results meets the project requirements.