EXTRACTION OF THE INDIVIDUAL TREE INFECTED BY PINE WILT DISEASE USING UNMANNED AERIAL VEHICLE OPTICAL IMAGERY

For eliminating pine trees infected pine wilt disease in southern China based on remote sensing technique, it is important to ensure the provision of timely information about individual diseased tree. It is not easy to detect and extract the diseased pine trees from conventional remote sensing techniques. This paper proposes a new approach for extracting information about individual diseased tree, without the use of satellite images and aerial hyperspectral images. Field measurements in different leaf infected stages indicates the possibility of extracting diseased trees by using only the three regular bands, red, green and blue. VEG was selected and proved to be the optimal index in 12 vegetation indices from the three visible bands. Using the adaptive local threshold selection methods, VEG grayscale image pixels could be automatically segmented into the diseased trees region. Based on mathematical morphology, the accuracy of individual tree information extraction reached 90%.


INTRODUCTION
Pine wilt disease (PWD), caused by Bursaphelenchus xylophilus, the pine wood nematode (PWN), is one of the most dangerous and destructive disease to forests in the world. It is known as the cancer of pine trees because of its wide transmission path, high speed, difficult control and high cost. The PWN, which is native to North America, can cause severe tree wilt and ultimately kill pine trees in forty days under suitable climatic conditions. PWD has invaded many countries in Asia and Europe. In china PWD was observed at first in Nanjing in 1982 (Wang, 2013). At present, the disease has spread to 16 provinces such as Zhejiang, Anhui, Fujian, Hubei and Sichuan, and continues to spread westward and northward, killing more than 500 million pine trees, destroying 333,300 hm 2 of forests and causing direct economic losses of hundreds of millions of yuan. The spread of PWD in China directly threatens the security of 33 million hm 2 pine forests and key ecological areas in southern China.
Cutting, fumigating and burning are effective methods to control the spread of PWD. Therefore, how to detect infected pine as early as possible is great significance to prevent the spread of PWD. The traditional field survey technique requires a large amount of labour cost and time consumption, especially in the mountainous and hilly areas in southern China. In fact, because of the visual deviation of investigators, these field survey data are often inaccurate.
Remote sensing technique has the characteristics of long distance, large area and indirect contact, and has been widely used in forest monitoring. Currently, this technique is used to carry out PWD detection, mainly using satellite images (Lee, 2006), (Takenaka, 2017) and aerial hyperspectral images (Pan, 2015), (Qin, 2016). Unfortunately, under the cloudy and rainy conditions in southern China, it is difficult to obtain highresolution and large-area images of these two types in time, and the timeliness of the observation data cannot be guaranteed. By using Unmanned Aerial Vehicle (UAV), high-resolution image data of large-area can be acquired in real time, which can effectively ensure the timeliness of remote sensing data application.
A new approach for extracting the individual tree infected by PWD using adaptive local threshold image segmentation method and mathematical morphology based on Vegetation Indices (VI) from UAV high-resolution optical image is proposed and evaluated in this paper. The rest of the paper is organized as follows. In Section 2, the materials and methods based on UAV optical imagery are introduced in the spectral characteristics, threshold image segmentation and individual tree extraction. The some practical details used to derive the individual trees are presented in Section 3. And the results are also presented. Finally, Section 4 gives the conclusions.

Experimental Site
The experimental site was located in Fushan Town, Fushun County, Zigong city, Sichuan province, which is in the southwest of Sichuan Basin and the upper reaches of the Yangtze River ( Figure 1). The terrain of the town slopes downward from north to south, and the terrain is dominated by hills, accounting for more than 60% of the total area. The land cover type here is diversified, which is a typical agroforestry ecotone. Among them, the cultivated land area is 1799hm 2 , dominated by upland crops, and the woodland area is 1666.67hm 2 , dominated by coniferous forests, with 38% woodland coverage. Pine wood nematode disease kills nearly 10,000 pine trees in this area every year, and hundreds of thousands of pine trees have been cut.

Platform and Sensor
On September 30, 2018, the researchers conducted the mission using the UAV platform DB-2AP101, which is capable of autonomous long-range flight with the help of an integrated GPS receiver and navigation system. The UAV platform has a maximum payload about 5kg, a maximum flight height about 4000m. The mission has a height of 600 meters and an area of 31 km 2 . UAV RGB images were acquired based on Nikon D800 camera. The sensor type of the camera is CMOS, the full width is 35.9mm × 24mm (4912 × 7360 pixels), and the effective pixels are 36.3 million. These images have a resolution of 0.15 meter.  The detailed process flow of the method presented in this paper is shown in Figure 2. The process is carried out as follows.

Spectral Characteristics:
According to the stage of the disease, the trees sampled included four types: healthy, early, middle and late. A SVC-HR-1024 spectrometer was used to measure the reflectance spectra of these leaves. The wavelengths sampled ranged from 350 nm to 2500 nm, with sampling intervals of 2 nm between 440 nm and 830 nm. Each sample was collected repeatedly for 3 times, and the average value of multiple measurements was taken as the reflectance spectra of the sample leaves. The total reflectance of these leaves from different stages is shown in Figure 3. The ordinary airborne camera has three imaging bands of red, green and blue, with the wavelength range of 600 ~ 700nm, 520 ~ 600nm and 450 ~ 520nm respectively. As can be seen from Figure3, in the visible band (about 350-700nm), the reflectance difference between diseased leaves and healthy leaves is small in the early stage, while in the middle and late stage of the disease, the spectral characteristics in the visible range begin to change significantly, mainly manifested by the obvious disappearance of absorption valley in the range of red light. This is because of severe water loss in diseased pine trees, which causes the leaves to wilt and have tended to be nonvegetative state in spectral characteristics.  The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2020, 2020 XXIV ISPRS Congress (2020 edition) (G) and blue (B). Based on the spectral characteristics mentioned above, the trees infected with PWD can be extracted by using the VI of visible bands. A set of 12 VI were calculated from the three visible bands of the UAV images. Table 1 gives information about the VI selected.

Threshold Image Segmentation:
After calculating the grayscale images, a threshold can be set to distinguish the vegetation and non-vegetation areas. There are many ways to set the threshold, either manually or automatically. In view of the inconsistency of remote sensing images, automatic threshold image segmentation is usually used. There are many threshold image segmentation methods, which can be divided into the global threshold selection methods and the adaptive local threshold selection methods. This paper adopts two common global threshold selection methods: Otsu method (Otsu, 1979) and Kittler method (Kittler, 1986), and two common adaptive local threshold selection methods: Bernsen method (Bernsen, 1986) and Niblack method (Niblack, 1986).

Individual Tree Extraction Method:
Ultimately, the detection results must include an accurate geographic location of the individual tree infected with PWD in order to cutting, fumigating and burning. However, after threshold image segmentation, there are other field objects, binarization noise and regional discontinuity in image. To solve these problems, it is necessary to process binary images. This paper presents an image processing method of extracting information about individual tree based on mathematical morphology (Jones, 1996). This method is based on two distinct properties of tree crown and carried out as follows. (1) The tree canopy is generally approximately circular, with a diameter of no more than 2 m (10 pixels) and no less than 0.5 m (3 pixels). Based on this property, the structural elements of individual tree in the image can be constructed to remove other field objects. At the same time, we can eliminate the false objects formed by binarization noise. (2) The information about the tree canopy edge is not important. The most important information is the location and number of the diseased trees. Based on this property, closing operation is adopted to connect the broken adjacent pixels and fill the hole in image segmentation results. This can eliminate the case where one diseased tree is labeled as multiple diseased trees due to the discontinuous segmentation results. After closing operation, the location information would be from the central point of the region.

VI Comparison
From the acquired images, we selected a representative image for analysis. In this image, there are some shadows caused by the tugged terrain and some uneven illumination areas. This image representation is common in southern China. By calculating twelve vegetation indices, 12 grayscale images were obtained (Figure 4).

RGB
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2020, 2020 XXIV ISPRS Congress (2020 edition)  As we can be seen from Figure 4, there are two red circles on RGB image, with two trees in each red circle. Left one is zone 1, bottom one is zone 2. By comparing the 12 kinds of VI grayscale images and Table 2, we found that some VI grayscale images could not reflect the distribution of diseased trees due to the uneven illumination from the sun during image acquisition. For example, BGI, GBVI, ExG, ExGMR, CIVE and MGBVI cannot directly detect the diseased trees in the two red circles by visual identification. In addition, some indices are too narrow dynamic range to keep enough fine details for extracting individual diseased tree, such as GRVI, MGRVI, RGBVI and VDVI. In the end, only the grayscale images of RGI and VEG could identify the diseased trees in the two red circles, while maintaining good contrast and details.

Threshold Image Segmentation Method Selection
After comparison of the 12 kinds of vegetation indices, the grayscale images of RGI and VEG were selected for threshold image segmentation. According to the threshold calculated by using Otsu, Kittler, Bernsen and Niblack methods, these binarization images are automatically generated ( Figure 5 and Figure 6). The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2020, 2020 XXIV ISPRS Congress (2020 edition) Figure 6. Comparison of four threshold segmentation results in VEG grayscale image (from top to bottom: Otsu method, Kittler method, Bernsen method and Niblack method).
By comparing Figure 5 and Figure 6, we can see that the segmentation results in VEG grayscale image are better than that in RGI grayscale image. In addition to the Otsu method, the VEG image segmentation results by using the other three methods clearly show the segmented pixels in the red circle regions at the top and bottom of this binary image. However, none of the four methods can produce similar results from VGI greyscale image.
In Figure 6, we can see that Otsu method has the worst results, and the results obtained by Kittler method have less details than those obtained by the two adaptive local threshold selection methods. The adaptive local threshold selection methods are better than the global threshold selection methods. Alternatively, the results of Bernsen method are similar to those of Niblack method. Similarly, the same conclusions can be observed from Figure 5.

Individual Tree Information Extraction
After VI comparison and threshold segmentation method selection, we determined the binary image generated by combining VEG image with Niblack method to extract the information about individual diseased tree. In this experiment, according to mathematical morphology, we created a circular structure element with a radius of 5 pixels. Then, the initial information about individual diseased tree was extracted by using closing operation. Finally, we discarded the small regions (less than 40 pixels) and generated the location information about individual diseased tree based on the central point of the region. The results are shown in Figure 7. Using the proposed method, 58 diseased trees were detected and extracted in this image. However, through our visual identification and ground surveying, the truth is 54 diseased trees. In Figure 7, we can see that the proposed method resulted in the omission of one diseased tree, as shown in the solid blue circle. Additionally, the proposed method also led to the 5 trees error, as shown in the orange dotted circle.

CONCLUSIONS
In this paper, we have proposed a new approach to extract the information about the individual tree infected by PWD for ecological protection in the mountainous and hilly areas in southern China, without the use of satellite images and aerial hyperspectral images.
On the analysis of the field measurements in different leaf infected stages, we found that there are significant differences in spectral characteristics between diseased and healthy pine tree, especially in the red light band. This phenomenon indicates the possibility of extracting diseased trees by using only the three regular bands, red, green and blue.
Comparing 12 kinds of VI calculated by R, G and B band, VEG was selected and proved to be the optimal vegetation index. And RGI is the secondary selection.
In order to automatically extract the information about diseased trees in the image, we experimented two types of threshold selection methods, and found that the adaptive local threshold selection methods are better than the global threshold selection methods.
In order to automatically obtain the information of diseased trees in the image, we experimented two types of threshold image segmentation methods, and found that the adaptive local threshold selection methods are better than the global threshold selection methods. However, the performance comparisons between Densen, Niblack, or other adaptive local threshold selection methods are worth further investigating.
Based on mathematical morphology in image, the accuracy of individual tree information extraction reached 90% in this experiment. In one case, there are fewer than the actual quantity due to the adjacent trees merged by closing operation. In another case, the phenomenon of false objects in the shadow area needs further analysis.