AN EFFICIENT WEED DETECTION PROCEDURE USING LOW-COST UAV IMAGERY SYSTEM FOR PRECISION AGRICULTURE APPLICATIONS

The use of Unmanned Aerial Vehicle (UAV) imagery systems for Precision Agriculture (PA) applications drew a lot of attention through the last decade. UAV as a platform for an imagery sensor is providing a major advantage as it can provide high spatial resolution images compared to satellite platform. Moreover, it provides the user with the ability to collect the needed images at any time along with the ability to cover the agriculture fields faster than terrestrial platform. Therefore, such UAV imagery systems are capable to fit the gap between aerial and terrestrial Remote Sensing. One of the important PA applications that using UAV imagery system for it showed great potentials is weed management and more specifically the weed detection step. The current weed management procedure depends on spraying the whole agriculture field with chemical herbicides to execute any weed plants in the field. Although such procedure seems to be effective, it has huge effect on the surrounding environment due to the excessive use of the chemical, especially that weed plants don’t cover the whole field. Usually weed plants spread through only few spots of the field. Therefore, different efforts were introduced to develop weed detection techniques using UAV imagery systems. Though the different advantages of the UAV imagery systems, they systems didn’t draw the users interest due to many limitations including the cost of the system. Therefore, the proposed paper introduces a new weed detection methodology from RGB images acquired by low-cost UAV imagery system. The proposed methodology adopts detecting the high-density vegetation spots as indication for weed patches spots. The achieved results showed the potential of the proposed methodology to use low-cost UAV imagery system equipped with lowcost RGB imagery sensor for detecting weed patches in different cropped agriculture fields even from different flight height as 20, 40, 80, and 120 meters. * Corresponding author


INTRODUCTION
Starting from mid-1980's, the concept of smart farming or precision agriculture (PA), has been raised in the agriculture industry as a management system. later, during the last two decades, the PA was considered as one of the top ten revolutions in the agriculture industry (Crookston, 2006). PA is a management system that aims to optimize the use of any inputs to the agriculture process as water, chemical herbicide, fertilizers, seeds, etc. to enhance the quality and quantity of the field's output while protecting the surrounding environment from any harm that might be caused due to the excessive use of these inputs (Zhang and Kovacs, 2012). To perform such smart management system, it is important to collect different information and process them to make the farmer able to take the right decision at the right time for the right spot of the field (Mulla, 2012). Therefore, different technologies as Remote Sensing (RS) and GPS/GNSS navigation systems have been used either as a source for information or as a tool for implementing different PA activities. For example, RS technology showed huge ability to provide valuable information for the farmer through using satellite or airborne platforms for different imagery systems. Such systems acquire images covering large areas within short time (Zhang and Kovacs, 2012). Then, the farmer can use the collected images to monitor the crop growth, crop stress, or predict the crop yield. Although such imagery systems are providing huge ability for farmers, they are limited with the low spatial resolution. Therefor, terrestrial RS systems were used to provide more detailed imagery data. Unfortunately, such systems are also limited with the needed time to cover the large agriculture fields. Therefore, there was a need for alternative platform that can fit the gap between aerial and terrestrial RS systems. Unmanned Aerial Vehicle (UAV) showed the potential to fit such gap. Through the last few decades the use of UAV as a platform for different sensors as cameras, lidar, GNSS, and IMUs proved its ability to be used for different applications, especially with its multiple advantages (Nex and Remondino, 2014). Generally, the UAV imagery system is considered as a low-cost alternative for other Remote Sensing systems that use satellite or aerial platforms. Moreover, UAV imagery systems are capable to provide higher spatial resolution as UAV flies at lower altitude compared to satellites and other aerial platforms. Also, UAV platforms provide the user with the ability to collect the imagery data with high temporal resolution which can enhance the flexibility of the data acquisition process. Therefore, UAV imagery systems are used for different PA applications as crop health monitoring (McCabe et al., 2015), weed management (Hassanein and El-sheimy, 2017;, and crop row detection (Slaughter et al., 2008). Weed management is one of the PA applications that UAV showed great potential to provide different solutions. Generally, weed is any wild plant that grows in the agriculture field which competes with the cultivated plant over the available resources in the field as water, fertilizers, growth space, and even sunlight (Hassanein and El-sheimy, 2017). Therefore, it is important to remove these wild plants as soon as possible to make the cultivated plants able to have the suitable amount of inputs to enhance the quality and quantity of the field's output. The process of controlling the existence of weed plants in the field is called as weed management. A conventional weed management system depends on spraying the whole field with a uniform distribution of chemical herbicides to kill the weed plants. As weed plants can't cover the whole field, such management approach is non-economic. Moreover, the excessive use of the chemical herbicide could harm the surrounding environment. Therefore, a PA management system was needed for better weed management which is called as Site-Specific Weed Management (SSWM) (Peña Barragán et al., 2012). In general, the SSWM process aims to detect the weed plants spots, or patches in the agriculture field then provide the farmer with a map for the detected weed, where the main inputs are images acquired using imagery system. As the major step of any SSWM process is to detect the weed plants in the agriculture field, different techniques have been proposed using satellite imagery systems  or terrestrial imagery systems (Tellaeche et al., 2008). Moreover, as UAV imagery systems showed potential to fit the gap between aerial and terrestrial imagery systems, different techniques were also introduced for the use of UAV imagery system for weed detection (Hassanein and El-sheimy, 2017;Mulla, 2013;Peña and Gutiérrez, 2012;Torres-Sánchez et al., 2013). Generally, most of these approaches depend on image processing techniques along with the spatial geometric characteristics of the crop field to detect weeds. These approaches use the high spatial resolution images acquired by the UAV imagery systems to detect any geometry change that can lead to detect weeds. For example, Göktoǧan et al. (2010) developed a UAV imagery system that collect images for the fields. Then, the authors used a vegetation index to discriminate between vegetations and the bare soil. As the authors developed the proposed system to be implemented in empty fields, any detected vegetations in the images were considered as weed plants. Although such system can't be practical for regular agriculture field that has crop and weed, it proved the potential of using UAV for weed management, weed surveillance, and remote control the spraying treatment of the weed. Another effort for using UAV imagery system for weed management was proposed by Jurado Expósito et al. (2009). The authors used the vegetation density along with the elevation information of the vegetation objects in the field to detect the weed plants. The achieved results showed that using the geostatistical techniques along with the elevation data were able to improve the detection of weed in the agriculture fields. Although such results, the main limitation is the need to generate the elevation model from the acquired images which might be time consuming process. Moreover, many other approaches depend on using the crop raw geometry which was originally suggested for weed detection since the late 1990's (Zwiggelaar, 1998). The general workflow of such approach goes through three general steps. First, a vegetation segmentation process is performed to differentiate between vegetation and non-vegetation objects and generate a binary vegetation image (Hassanein et al., 2018). Then, the crop rows are detected using different linear feature detection techniques. Finally, the vegetation objects between crop rows are classified as weed plants. The main difference between these approaches depend on the used methodology of detecting the crop rows or the vegetation segmentation methodology. For example, Peña Barragán et al. (2012) used a UAV imagery system to collect imagery data for an agriculture field where seeds were planted in rows. The images were acquired at different dates and flight heights using multispectral imagery sensor. Their approach first detects the crop rows, then it classifies the in-between rows objects into vegetation and nonvegetation objects. Such classification or discrimination depends on using the Normalized Difference of Vegetation Index (NDVI) to detect the plants and any undetected object will be considered as bare soil. The main contribution in the authors work was to develop an Object Based Image Analysis (OBIA) approach to detect the crop rows. The achieved results showed a 95% accuracy of crop row detection and 90% accuracy of weed plants detection. Later, following the same track, Peña & Gutiérrez (2015) proposed another weed detection approach. The authors attached two different imagery sensors, visible light and multispectral, to UAV platform and use that imagery system to collect imagery data for part of an agriculture field infected with weed plants at different heights (30 and 100 meters). Using different imagery sensors and flight heights cases was useful to study the effect of changing of the sensor and the flying height on the weed detection accuracy. The proposed methodology depends on discriminating the vegetation objects using NDVI vegetation index for the images acquired by the multispectral sensor and the Excess Green Vegetation Index (ExG) for images acquired by the visible light sensor. Then, the system detects the crop rows using the Hough Transform linear feature detection method (Duda and Hart, 1972). Finally, the vegetation objects between rows are classified as weed plants. The achieved results showed the potential of using such system to detect weed plants. The main conclusion recommended to use multispectral imagery sensor at 30 meters flight height to achieve high weed detection accuracy. Another weed detection approach using the crop row geometry was proposed by Peña Barragán et al. (2015). The authors provided a detailed study for using a UAV imagery system for weed detection. The proposed methodology was tested at different flying heights as 40, 60, 80, and 100 meters along with collecting the imagery data using two different sensors which are a visible-light camera and a multi-spectral camera to evaluate the change of imagery sensor effect. The proposed system used a similar OBIA image analysis technique, which was introduced by the same authors in earlier publication (Peña Barragán et al., 2012). The OBIA technique was used to detect the crop rows in the collected images. Then, the objects between rows were classified into vegetation and nonvegetation objects using vegetation indices. The vegetation objects detected between the crop rows were considered as weed plants. The achieved results concluded that using the multispectral camera to collect imagery data with flying height of 40 meters is the best combination to reach the highest accuracy of 91 % for weed detection. Although the high accuracy of weed detection proposed by the different techniques, there are still some limitations that affects the use of UAV imagery systems for PA applications and weed management is one of them (Erickson and Widmar, 2015). These limitations include the cost of the system, the recommended flight height, the payload, and the limited battery capacity. Usually, the multispectral cameras are preferred to be used for different PA applications, including, the weed management. Such preference is mainly due to the ability to discriminate between vegetation and non-vegetation objects easily using vegetation indexes as NDVI concluded from the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-1, 2018 ISPRS TC I Mid-term Symposium "Innovative Sensing -From Sensors to Methods and Applications", 10-12 October 2018, Karlsruhe, Germany multispectral channels in these cameras, though there is a need to reduce the cost of such systems to encourage farmers to use them. Therefore, improving the use of low-cost visible light or RGB camera as the sensor for UAV imagery system might be a suitable choice. Moreover, the preference of low flight height is mainly to get high spatial resolution. Such resolution is needed to be able to detect every weed plant. Though, such low flight height causes a major limitation for using UAV imagery systems for weed detection. The main limitation is the huge amount of imagery data generated from acquiring the images at these low flight heights. This number of images needs large processing time which affects the efficiency of the weed detection procedure. Therefore, the proposed paper introduces a new weed detection methodology that adopt two main concepts to enhance the use of UAV imagery systems for weed detection. First, the proposed methodology aims to detect the weed patches instead of detecting individual weed plants. Through detecting the weed patches, the UAV imagery system can fly at higher flight height as the needed spatial resolution to show the weed batch is lower than the needed spatial resolution to detect every plant in the field. Second, the proposed methodology adopts the use of lowcost RGB cameras as the imagery sensor. To enhance the use of RGB cameras, the proposed system will implement vegetation segmentation technique that doesn't depend on the regular vegetation indices that are used for RGB images. Through the following parts the proposed weed detection methodology is described. Then, section (3) provides a detailed description of the data acquisition process along with the methodology verification procedure. Later, section (4) provides an analysis for the achieved results. Finally, the paper main conclusions are presented in section (5).

METHODOLOGY
The main objective of the proposed methodology is to generate a weed image using one main input which is RGB image for the agriculture field acquired by a low-cost UAV imagery system. A weed image is an image that represents one type of objects which is weed patches. Therefore, the methodology classifies the objects in the acquired RGB image into two main categories which are the weed patches and the non-weed patches. The objects that belong to the first category, which are the weed patches, will be presented with white color in the generated output which is the weed image. Adopting the concept of detecting the weed patches instead of detecting every weed plant was mainly motivated for two main reasons. First, the weed plants are generally distributed in one of three forms in the agriculture fields, as shown in figure (1), where the patchy weed distribution is considered as the most common distribution in the agriculture field (Wiles et al., 1992). Second, detecting the weed patches instead of detecting the weed plant will reduce the needed spatial resolution for the detection. Therefore, the recommended flight height will increase to be more than the 40 meters recommended by other weed detection approaches . Another important note, the proposed methodology depends on generating a grid that will be used to classify the objects in the input RGB image into weed patches and non-weed patches. As the weed patches increase the vegetation density, the generated grid will compute the vegetation density of each block in the grid. Then, the blocks with high vegetation density percentage will be classified as weed patches. Figure (2), shows the general flowchart of the proposed methodology which covers 5 main steps as will be described in the following subsections.

Vegetation Segmentation
After collecting the RGB image for the agriculture field, the proposed methodology performs a vegetation segmentation process that aims to convert the RGB image into vegetation binary image, where the vegetation objects, as crop and weed plants, are presented with white color, while the non-vegetation objects, as soil, are presented with black color, as shown in figure (3), (Hassanein et al., 2018). The main motivation for performing a vegetation segmentation process is to discriminate the vegetation objects from the remaining objects in the acquired image, which will add more efficiency for the proposed methodology. Also, the output of this process will be used to classify the detected vegetation objects into weed patches and non-weed patches. Different methodologies and techniques were proposed for vegetation segmentation process from RGB images (Hamuda et al., 2016;Hassanein et al., 2018;Torres-Sánchez et al., 2015), and based on comparison studies in different publications, the proposed methodology used a vegetation segmentation process that depends on converting the color space of the collected The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-1, 2018 ISPRS TC I Mid-term Symposium "Innovative Sensing -From Sensors to Methods and Applications", 10-12 October 2018, Karlsruhe, Germany image from RGB color space into Hue, Saturation, and Value (HSV) color space. The main motivation for using the HSV color space is to use the Hue channel image which has the ability to represent the color of the objects in the image without any illumination effects (Hassanein et al., 2018). Therefore, there are two main steps in the used methodology to generate the vegetation binary image. First, the color space of the image is converted from RGB into HSV. Then, the Hue channel image is extracted, which represent a grayscale image where colors are represented without any illumination effects. The second step depends on detecting a suitable threshold value to discriminate between the vegetation, or the green color, from other color. Such threshold value is detected using published vegetation segmentation methodology that depends on analysing the histogram of the Hue values of the generated Hue image. Generally, as any image for agriculture field should contains two major colors which are the green for the vegetation objects and the non-green for the non-vegetation objects, so through analysing the histogram peaks, the suitable threshold value that discriminate between these colors could be detected (Hassanein et al., 2018). Finally, the threshold value is used to classify the color values in the Hue image into white for the green or the vegetation objects and black for the non-green or the non-vegetation objects.

Detect Suitable Grid Size
As mentioned, the proposed methodology depends on detecting the spots with high vegetation density to be considered as weed patches. Therefore, there is a need to generate a grid of blocks to be used to judge the vegetation density at each spot in the agriculture field's image. As the proposed methodology doesn't request a specific flight height for the UAV imagery system and it should work for different types of crops with different crop rows interval distances, the suitable size of each block in the grid might change from image to image. Therefore, the proposed methodology defines the suitable size of each block in the grid through dividing the image with n*n sections, as shown in figure (4) as an example for 4*4 grid. Moreover, as the proposed methodology depends on comparing the vegetation densities to detect the weed patches, the suitable value of (n) is considered as the highest number that makes all the blocks have vegetation existence.

Vegetation Density of Blocks
The following step is to compute the vegetation density of each block in the generated grid. Such step could be easily computed as the shown in equation (1). Though, to speed the computation process, the concept of integral image is implemented (Bay et al., 2008).
Vegetation Density = sum of white pixels in the block / number of pixels in the block (1) To speed up the computation of the system, the integral image is used. As shown in figure (5), finding the sum of white pixels for each block using the integral image is providing an advantage for the system. Such advantages of low time consumption and low amount of processed data are the main motivation for using the concept of integral image for different image processing approaches as SURF feature detection (Bay et al., 2008). Figure 5. The sum of elements in the hatched block can be computed from one simple equation using the integral image (Bay et al., 2008).

Detect Weed Density Threshold
The next step of the proposed methodology is detecting the suitable vegetation percentage that should be used as threshold to classify each block either as weed block or non-weed block. Such threshold value is detected through, first categorizing the vegetation density percentage of each block into one of ten levels. For example, if vegetation density percentage of a block is between 0% up to 10%, this block will belong to level 1, if vegetation density percentage of a block is between 10% up to 20%, this block will belong to level 2. The second step, based on the number of block for each level, the suitable threshold level can be detected, especially that blocks with weed patches will have odd vegetation density compared to the regular vegetation densities of other block. Such assumption is concluded as the vegetation objects are equally distributed in the agriculture field. Therefore, there should be a gap between the vegetation density of weed blocks and the non-weed blocks. Such gap value is used as the threshold value to discriminate between weed and non-weed blocks.

Weed Image Generation
The last step of the proposed methodology is to generate the main output of the whole process which is the weed image. Generally, the weed image combines all the blocks that their vegetation components are considered as weed patches. Then the vegetation objects in these weed blocks are represented with white color, while the remaining parts of the image are black.

METHODOLOGY IMPLEMENTATION
To evaluate the proposed methodology, a low-cost UAV imagery system was used to collect multiple RGB images for two agriculture fields. The following subsection will provide a detailed description for the used UAV, imagery sensor, cropped fields, and the collected images. Then, some of the achieved weed images using the proposed methodology will be provided.

Data Acquisition
As mentioned, to implement the proposed methodology different RGB images were collected using low-cost UAV imagery system. As shown in figure (6), the used UAV is the quadcopter Inspire 1 from DJI. The used UAV is equipped with the DJI Zenmuse X3 RGB camera. The described system was used to collect different images for two agriculture fields. The first field is planted with canola, while the second field was planted with beans. Seven different images were acquired for the two fields at different spots, and flight heights. The collected images provided different cases of flight height, as shown in table (2), and weed densities, as shown in figure (7). Beans 20 4 Beans 20 5 Beans 80  6 Beans 120 7 Beans 20 Table 1. List of the used images to evaluate the proposed methodology

Results
Every RGB image goes through the proposed steps of the described methodology. Figure (7), shows the inputs and the outputs for the methodology for all the tested images, while figure (8), shows a sample of the achieved results for every step of implementing the methodology on image # 2.
6 (a) 6 (b) The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-1, 2018 ISPRS TC I Mid-term Symposium "Innovative Sensing -From Sensors to Methods and Applications", 10-12 October 2018, Karlsruhe, Germany 7 (a) 7 (b) Figure 7. Generated weed images from the proposed methodology.

RESULTS ANALYSIS
The achieved results, as shown in figure (7), proved the ability of the proposed methodology to detect weed patches in the agriculture field using the RGB images. Moreover, at different heights, the weed patches were detected as shown in cases (5) and (6) in figure (7). Furthermore, the methodology was capable to indicate if the image of the field contains a large weed patch or just small scattered of weed plants as shown in case (7) in figure (7), as the weed image as totally black which means that the system didn't detect any weed patches. Also, the methodology detected weed patches even from images acquired at 80 and 120 meters flight height. Although the methodology couldn't detect all the weed patches, it proved the potential of the system. The main reason for such limitation is due to the low quality of the generated vegetation binary image generated from the vegetation segmentation process, which was affected with the low resolution of the images at such height.

CONCLUSIONS
The proposed paper provides a new weed detection methodology that can be used as part of a smart weed management system. Such system can be used for enhancing the use of UAV imagery system for important PA application which is the Site-Specific Weed Management (SSWM). Generally, the proposed weed detection methodology depends on detecting the weed patches from RGB images. The use of RGB imagery sensor will provide an advantage to the system as such sensors are low-cost compared to the use of multispectral imagery sensors, which are currently preferred by other PA applications. The proposed weed detection methodology depends on detecting spots with high vegetation densities in the acquired image. These detected spots are considered as weed patches. Therefore, a grid of blocks is generated to cover the full image. Then, based on comparing the vegetation densities of the blocks, the methodology detects every block with extreme vegetation density compared to other blocks. Later, the proposed methodology was tested through using low-cost UAV imagery system equipped with RGB imagery sensor. Different RGB images were acquired for two agriculture fields at different flight height. Based on the achieved weed images, the proposed methodology proved its ability to detect weed patches. Moreover, the methodology was able to detect the weed patches at the different height, even with less detection quality. Furthermore, the methodology was capable to state if there are weed patches or not in the agriculture field. Also, although the methodology showed high potential for detecting weed patches even from images collected at 120 m flight height, the quality of the detection process faced some limitations. The main reason for such limitation is due to the low quality of the generated vegetation binary image because of the low resolution of the acquired images at that height. Finally, based on the achieved results, it is recommended as future work, to work on enhancing the quality of the weed detection process at high flight height. Moreover, such weed detection process could be used as the first step to generate a weed map which provide the user with the positions of the detected weed patches.