UAV-based Crops Classification with joint features from Orthoimage and DSM data

Accurate crops classification remains a challenging task due to the same crop with different spectra and different crops with same spectrum phenomenon. Recently, UAV-based remote sensing approach gains popularity not only for its high spatial and temporal resolution, but also for its ability to obtain spectra and spatial data at the same time. This paper focus on how to take full advantages of spatial and spectrum features to improve crops classification accuracy, based on an UAV platform equipped with a general digital camera. Texture and spatial features extracted from the RGB orthoimage and the digital surface model of the monitoring area are analysed and integrated within a SVM classification framework. Extensive experiences results indicate that the overall classification accuracy is drastically improved from 72.9% to 94.5% when the spatial features are combined together, which verified the feasibility and effectiveness of the proposed method.


INTRODUCTION
Information on crop sowing type and yield is an important basis for the country to formulate scientific agricultural policies and economic plans, and is also an important part of the core indicators of agricultural statistics in countries around the world.Timely understanding and mastering crop types has important practical significance for accurately estimating and predicting crop yields, strengthening crop production management, adjusting agricultural cropping structure, and ensuring national food security.In addition, the acquisition of high-precision crop acreage has gradually become one of the most important scientific issues in agricultural land system research (Cao Weibin et al, 2004a;Huang Qing et al, 2009a;Yang Bangjie et al, 1997a).For a long time, China's agricultural crop sowing types, sown area, planting quantity and other important agricultural statistics have been obtained mainly through comprehensive statistical methods or sample surveys those have been reported step by step.Agricultural survey teams distributed throughout the country regularly collect planting areas and types of crops.Crops growth and disaster-affected situations are reported step by step or reported directly to the Ministry of Agriculture as the basis for analysing the state of agricultural structure planting and taking countermeasures.The traditional method is to manually measure the ground sample or use GE images to assist in the investigation of crop classification (Liu Jia et al, 2015a).Because the surveyors' abilities to investigate are different and cannot be objectively standardized, there are lags in the collection, processing, and reporting process.Differences in the information, and this method has many defects such as huge investigation workload, high financial and material costs, and long investigation period (Yang Bangjie et al, 1997a).In recent years, remote sensing technology has played an important role in dynamic information extraction of crop areas and crop distribution mapping (Zhang Jiankang et al, 2012a).Satellite remote sensing information has the characteristics of large coverage area, short detection period, abundant data, low cost, and provides new technical means for quickly and accurately obtaining crop types (Chen Zhongxin et al, 2016a).However, due to the long period of high-resolution satellite re-entry cycle, the data for a given area at a given time cannot be guaranteed.The accuracy of crop land area monitoring using satellite remote sensing cannot meet the requirements and must be supplemented by ground sample surveys and ground sample survey data.Using the ground sample survey data to calculate the deduction coefficient of linear and fine features, to achieve the fine extraction of crop planting areas extracted by remote sensing.The emergence and development of UAV remote sensing technology has provided new ideas for the collection of crop information (Freeman et al, 2015a;Mesas-Carrascosa et al, 2014a;Rokhmana et al, 2015a).At the small and medium scale, UAV remote sensing can play a greater role and can obtain more accurate crop distribution information, which is of great significance to the development and application of crop monitoring technology.UAV remote sensing has features such as high resolution, simple operation, fast data acquisition, and low cost.It can quickly perform image collection for a certain area and combine ground actual measurement data to quickly and accurately complete the crop planting information monitoring task.It can be used as a useful complement to satellite remote sensing and aerial remote sensing, and provides accuracy verification for large-scale remote sensing (Del Pozo et al, 2014a) At present, many scholars have done a lot of research on the classification of crops based on drones, and put forward many techniques and methods.Pena used object-oriented methods based on the obtained six-band multispectral images to achieve weed mapping during early corn growth (Pena et al, 2013a).Gini used the maximum likelihood method and used the obtained multispectral images to classify different trees (Gini et al, 2016a).Sona used multi-spectral images for soil and crop mapping to realize the application of multi-spectral data in precision agriculture (Sona et al, 2016a).Rypochi Doi performed color synthesis of multi-spectral images, and found that this method can increase the distinguishability of similar pixels (Rypochi Doi et al, 2015a).Oumer used multi-spectral images and RGB images to achieve land cover and crop classification using Random Forest Algorithm (Oumer et al, 2017a).Xiuliang Jin used an optical camera mounted on a drone to acquire RGB images and used the SVM algorithm to estimate the planting density of winter wheat.The maximum likelihood method and SVM algorithm were used to classify the shadows, indicating that the SVM classifier had a better classification effect in classification process (Xiuliang Jin et al, 2017a).Adrien Michez used drones to obtain multi-temporal and hyperspectral images and used a random forest algorithm to classify the tree types and health status of the riparian forest zone (Adrien Michez et al, 2017a).J. Senthilnath used the K-means generator and the expected maximum method to classify crop mapping and canopy mapping.An aerial image classification method based on feature selection was proposed to realize the recognition of flooded areas and roads (J.Senthilnath et al, 2017a).Han Wenting used multispectral cameras to achieve the extraction of field canal distribution information using an object-oriented feature combination hierarchical classification method (Han Wenting et al, 2017a).However, the use of multi-spectral devices has disadvantages such as high cost, low flight efficiency, small work area, and difficulty in matching with visible light images.Gao Lin realized the estimation of winter wheat leaf area index based on digital images of drones (Gao Lin et al, 2016a).Guo Peng used visible light images and selects brightness, saturation, and red second-order moments as the best classification features to classify crops, which is significantly higher than the color index method (Guo Peng et al, 2017a).Kim used fixed-wing drones to obtain orthophotos, multi-spectral images, and digital surface models, using a random forest approach to classify land cover types (Kim et al, 2017a).Wang Xiaoqin adopted the visible light band image and proposed a new visible vegetation index with visible light bands to achieve healthy green vegetation information extraction.The visible light band can be used as a data source for extracting crop planting information (Wang Xiaoqin et al, 2015a).However, from the above studies, it is found that the use of multi-spectral equipment is relatively expensive, the flight efficiency is low, the work area is small, and it is difficult to match the visible light image.Yang Qi adopted a low-altitude remote sensing platform to carry a high-definition digital camera and collected high-resolution digital images of sugarcane during the whole growth period.The CSMs at each growth stage were established and the plant heights were extracted with and without ground control points.The results showed that this method is feasible and the plant heights extracted by CSMs have high accuracy (Yang Qi et al, 2017a).Bendig conducted several studies based on a digital camera platform equipped with a drone to obtain barley plant height, confirming that the plant height extracted from CSMs has good accuracy, and established an estimation model of barley plant height and biomass (Bendig et al, 2014a).It can be seen that the height of crops extracted from visible bands can be used as a feature of crop classification.In this experiment, we aimed to compare and analyse the classification features of crop species by using high-resolution images of UAVs and digital surface models to obtain farmland cover classification maps.In particular, in this experiment, the filter processing was performed on the DSMs, and the filtered image was added to the classification process as the height features of the crop, thereby improving the classification accuracy.Finally, we compared and analysed the results of different combinations of RGB features, texture features, and height features.

AREA TEST AND DATA CAPTURE
The research area is the experimental base of Chinese Academy of Agricultural Sciences, located in Minzhu Township, Daowai District, Harbin.A variety of crops were planted in this research area, and the terrain is undulating.The data from this study was collected from the drone remote sensing test from August 3 to August 4, 2017.The ground control point data was marked and measured on August 3, 2017.The drone image data collection date is August 4, 2017 using Fixedwing UAV with Sony Digital Camera.

CROP CLASSIFICATION METHOD AND PROCESS
This experiment was based on spectral, texture, and spatial information and generated crop classification maps using the SVM classifier.The specific steps are shown in Figure 1.

The generation of DOM and DSMs
In Smart3D, Digital Orthophoto Map(DOM) and digital surface models(DSMs) were produced using POS data and ground control point data.The DOM that we chose had a spatial resolution of 10 cm and 7500*7500.

Feature Analysis and Combination
There are 12 types of cropland cover, including rice, corn, unripe wheat, ripe wheat, harvested wheat, soybeans, trees, grassland, bare land, roads, greenhouses and houses.After normalization, trained SVM models and predicted pixel labels in LIBSVM.Selected RBF kernel in this paper.The number of samples of each type of feature is shown in Table 3. Due to the lack of spectral bands and complex types of objects, a single feature cannot completely distinguish all crops.Through comparative analysis, we found that the second-order moment texture of the green band had a smaller variance and coefficient of variation, and can better distinguish between twelve different types of ground objects and can be used as an effective feature.Take the road as an example, calculate the eight features in the green band, and calculate the difference coefficient with the other eleven types of ground features(Table 4).

Height Feature Extraction Table 5. Comparison of roads and other features in DSMs features
By comparison and analysis, the features of DSMs filter, variance and contrast can better distinguish crop types and can be used as effective distinguishing features.During the processing, the DSMs contrast feature and the DSMs variance feature were calculated using the 9*9 and 27*27 processing window.Take the road as an example, calculate the eight features in the DSMs filters images, and calculate the difference coefficient with the other eleven types of ground features(Table 5).
After comparison and analysis from the above table, four kinds of feature combinations are selected: the first one is to use RGB features for classification, the second is to use RGB features and second moments of green band for classification, the third experiment is to use RGB features and DSMs for classification, the fourth experiment is to use RGB, the second-order moment of green band, and DSMs contrast and variance filter features for classification.

Image classification by SVM
In the case of limited sample information, support vector machine can well balance the complexity and learning of the model, and has a good generalization ability.Therefore, it has been widely used in remote sensing image classification research in recent years, and has achieved good effect.First, the test combines the features of the spectrum, texture, and height to determine the combination of features.The composite features are normalized, input SVM to train, and then use the particle swarm optimization and grid search to find the optimal parameters and crossvalidation.Finally, the classification model was used to classify different features and calculate the classification accuracy (Liu, 2017).The above figures show the classification results of various combinations of features: When using only RGB features for classification, many trees were mistakenly classified as corn, the shadow of the greenhouse was wrongly divided into greenhouses, and some buildings were mistakenly divided into roads.In the classification results with the addition of green band texture features, the shadows of many trees are mistakenly classified as roads.With the addition of altitude information, the shadows of trees and buildings are reduced, and the classification within the block is more consistent.After adding the green band and DSMs filtering, there are fewer fine points in the class, and there are some trees that are mistakenly divided into buildings on the edge.The variance and contrast filtering accuracy of the 27*27 addition window is 93.11% and Kappa is 0.92.Finally, after adding the second moment feature of the green band, the DSMs variance and the contrast filtering feature, the detection accuracy is 94.5% and Kappa is 0.94.Although the overall classification accuracy is highest after adding the second moment of green band, DSMs, DSMs variance and contrast features, the edges of the tree are misclassified into buildings, and the second moment of green band cannot provide effective improvement after adding DSMs.

CLASSIFICATION RESULT
The optimal feature combinations are RGB, DSMs, DSMs variance and contrast features.

CONCLUSION
In this paper, we use drones equipped with a digital camera to collect RGB images of the test site, and then those images are processed in order to generate an orthoimage and a DSM of the entire area.Differ to traditional UAV-based remote sensing methods which mainly rely on spectra features from the RGB images, spatial features extracted from the DSM data are emphasized in our method.We adopt a typical SVM classification framework where texture and spatial features extracted from the orthoimage and DSM data are combined together.Experiments results indicated that the overall classification accuracy is dramatically improved when the spatial features especially that of the altitude demission are introduced.
According to the experiments results, the contributions of spatial features were as follows: 1.Besides the overall classification accuracy, the homogeneity of each category was also improved at the same time; 2. Classes which are hard to separate in colour space, such as tree, grass and crops, became distinguishable to each other when features in the altitude demission was introduced.The reason is, height value and its variance of each class is different according to statistical analysis.3. Misclassifications caused by shades in the RGB images was largely reduced with the assistance of spatial features.Although classification accuracy was much improved with the proposed method, there are still tackle issues remained in this field.For example, misclassification occurred in the edge of the class due to the poor data quality of the DSM data.Usually at the edge of objects, noises of the height value increased sharply, which is quite common in such kind of data.Moreover, random noises can be seen in almost every category in the final classification results, because of a pixel by pixel classification approach was used.Further research will be conducted to solve those issues mentioned above.For example, number of features by neighbourhood analysis should be increased in order to overcome the drawbacks of pixel by pixel classification pattern.Furthermore, other classification approaches such as deep learning method should also be investigated in future.Finally, we also plan to add other sensors such as low cost multi-spectral sensor to increase the feature space for the sake of a better performance in classification accuracy of crops.

Figure
Figure 1.Research Process

Figure 2 .
Figure 2. DOM(left) and DSMs(right) feature extractionTexture feature is extracted by ENVI software.The mean, variance, homogeneity, contrast, dissimilarity, entropy, second moment, and correlation of RGB bands and DSM band are obtained.Then the mean, variance (D), coefficient of variation (V), and coefficient of difference between classes (  ) were statistically analyzed for each type of sample.Selecting the feature with a small coefficient of variation and a large difference in classification coefficient and combined RGB to form new training features.

Figure 6 .
Figure 6.Original image and RGB feature

Table 4 .
Comparison of roads and other features in green band features Feature index First of all, the test classification accuracy based on the support vector machine classification of the visible RGB band is 73.166% and Kappa is 0.70.Second, after adding the second moment feature of the green band, the accuracy of the classification accuracy is 83.38% and Kappa is 0.82.Third, the fourdimensional feature formed by adding DSMs test accuracy is 91.44% and Kappa is 0.90.Based on the RGB and DSMs bands, addition of DSMs contrast filtering and variance filter.The accuracy of the variance filter with window 9*9 added is 92.25% and Kappa is 0.92, and the accuracy of the variance filter with window 27*27 is 93.35% and Kappa is 0.93.The contrast filter test accuracy of the 27*27 window is 92.77% and Kappa is 0.92.