QUANTIFYING LODGING PERCENTAGE, LODGING DEVELOPMENT AND LODGING SEVERITY USING A UAV-BASED CANOPY HEIGHT MODEL

Unmanned Aerial Vehicles (UAVs) are increasingly used, and open new opportunities, in agriculture and phenotyping because of the flexible data acquisition. In this study the potential of ultra-high spatially resolved UAV image data was investigated to quantify lodging percentage, lodging development and lodging severity of barley using Structure from Motion techniques. The term lodging is defined as the permanent displacement of a plant from the upright position. Traditionally lodging quantification is based on observations that need, and vary with observers in the field. An objective threshold approach was proposed in this study to improve the accuracy in lodging determination. Across breeding trials, manual reference measurements and UAV based lodging percentage showed a very high correlation (R2 = 0.96). In addition, the multi-temporal lodging percentage development was used to estimate the recovery rate and to determine the influence of different lodging events. Based on the parameter lodging percentage an approach was developed that allowed the assessment of lodging severity, an information that is important to estimate the yield impairment. Lodging severity can be used for insurance applications, precision farming and breeder research. This trait, together with differentiated recovery are novel traits next to lodging severity that will aid the selection for genetic lines.


INTRODUCTION
The increasing digitalization in agriculture is caused by the rapid development in sensor technology and data processing (Atzberger, 2013;Siegmann and Jarmer, 2015). The use of unmanned aerial vehicles (UAVs) can help to advance and accelerate this process to phenotype plants in short time periods (Burkart et al., 2017;Gómez-Candón et al., 2014;Zhang and Kovacs, 2012).The versatile applications of UAVs in agriculture and other areas are caused by their low costs, simple handling and high flexibility (Eling et al., 2015;Grenzdörffer et al., 2008;Hodgson et al., 2016;Mancini et al., 2013). The possibility to acquire ultra-high resolution spatial UAV data in comparison to satellite and airborne systems, however, is a basic requirement to assess the three-dimensional (3D) canopy structure of crops using feature matching and Structure from Motion (SfM) techniques (Colomina and Molina, 2014;Dandois and Ellis, 2013;Turner et al., 2012). The 3D canopy structure acquired with a red, green and blue (RGB) camera was applied to assess three plant traits (lodging percentage, lodging development, lodging severity).
To deviate the canopy height from the canopy structure a nonvegetated ground model is needed. This ground model determines the top soil surface and is normally acquired via UAV overflight (Bendig et al., 2013;Chu et al., 2017). The potential of UAV derived canopy height were already evaluated in several studies (Anthony et al., 2014;De Souza et al., 2017;Stanton et al., 2017), in detail for multi-temporal growth curve generation Holman et al., 2016) or biomass estimation (Bendig et al., , 2014. Compared to the classical plant height measurements collected with a measuring ruler at a specific position, the UAV approach allows to derive the height of the complete canopy Bendig, 2015). Thus, the UAV based canopy height implied various height information in contrast to the plant height measurement in the field with a ruler, where usually only one measurement per plant is possible.
The canopy height can additionally be used to identify lodge areas. Lodging is defined as the permanent displacement of a plant from the upright position (Berry and Spink, 2012;Rajapaksa et al., 2018) and leads to qualitative and quantitative yield losses of up to 45 % (Berry and Spink, 2012;Peng et al., 2014;Pinthus, 1974;Weibel and Pendleton, 1964). The losses are mainly as a result of the lodging severity and the developmental stage of occurrence (Berry et al., 2004;Fischer and Stapper, 1987;Briggs, 1990). Extreme weather conditions like heavy rain, storm, excessive nitrogen and disease can cause lodging. This results in a growing need to select for genetic lines with greater lodging resistance (Pinthus, 1974). Using UAV data for the spatial assessment of lodging is a very suitable method to automate the detection of lodging and replace laborious and subjective ground data collection. Already Susko et al. (2018) tried to assess crop lodging with a field camera track system. Additionally, Yang et al. (2015) used polarimetric index from RADARSAT-2 data for monitoring wheat lodging. Liu et al. (2018) further used visible and thermal infrared images derived from UAV for rice lodging estimation. Also Murakami et al. (2012) quantified lodging in buckwheat using the 3D canopy structure. In this study, however, the area of lodging was determined by using a threshold at which canopy height lodging occurred, but the application of those thresholds applied in different studies (Bendig, 2015;Chapman et al., 2014;Yang et al., 2017) were defined by subjective inspections rather than by mathematical approaches. The main goal of the presented study is to show a new method using an objective threshold approach that enables the assessment of the lodging percentage without adjusted threshold and subjective decisions. Additionally, the approach can be used to determine the lodging development, the recovery rate of crops and evaluate the influence of different lodging events based on a multi-temporal consideration of lodging percentage. Navabi et al. (2006) already demonstrated on over 140 different wheat genotypes that the extent of recovery capability varied among genotypes. Similar results were found by Briggs (1990) for barley.
In general, the lodging percentage parameter is only a decision between presence and absence of lodging. However, the crop canopies can be affected by different lodging severities resulting in different amounts of yield losses. Different studies already investigated the influence of lodging severity related on yield (Berry and Spink, 2012;Fischer and Stapper, 1987;Michael, 1998;Murakami et al., 2012). Ground data based on visual lodging scores are generally insufficient in accuracy, efficiency, and objectivity (Murakami et al., 2012;Simko and Piepho, 2011). Until now, only Chu et al. (2017) tried to assess the lodging severity of corn field by quantifying the number of lodged plants. However, due to the different plant structure and plant density of corn, this approach cannot be applied for cereal crops. Therefore, in this study, a new method is presented that allows to assess the lodging severity of barley using information on how strong the canopy is affected by lodging based on the canopy height variation derived from UAV images.

Study Area
The study was conducted at Campus Klein-Altendorf agricultural research station of the university of Bonn (50°37ʹN, 6°59ʹE, altitude over sea level 186 m), Germany. The study site was an experimental setup consisting of several small breeder plots, each 2.62 × 3 m in size. The layout included three different summer barley (Hordeum vulgare) cultivars with two different sowing densities and six repetitions. The codes explained in Table 1 represents the relevant genotypes for the study. The high density (300 seeds m −2 ) reflected the common sowing density in Germany. The lower density consisted of 150 seeds m −2 . The selected barley cultivars varied in canopy characteristics and plant height. Sowing was done on 9th April, 2016.

Weather Conditions
The seasonal development of barley was influenced by environmental conditions recorded at a weather station in situ Campus-Klein-Altendorf. The heavy rain events ( Figure 2) especially in June and July influenced the plant development and resulted in a high amount of lodged plants.

UAV Platform and Sensor
For data acquisition the Falcon-8 UAV (Ascending Technologies GmbH, Krailing, Deutschland) and a Sony (Sony Europe Limited, Weybridge, Surrey, UK) Alpha 6000 RGB camera (24 megapixel, 6000 × 4000 pixels) were used. The RGB camera was integrated on a gimbal ( Figure 3). Pitch and roll movement of the UAV was balanced and images were acquired according to a planned waypoints pattern with 60% cross and 80% forward overlap. Depending on the weather conditions the flight duration varied between 10-15 mins.

Data Processing
Structure from motion (SfM) algorithms were used for processing the UAV images in Agisoft Photoscan (Agisoft LLC, Saint Petersburg, Russia, version 1.4.1). The algorithms identifies corresponding images by feature recognition (Agisoft, 2018). Via a certain number of overlapping images, it recreates their orientation in a spatial three-dimensional (3D) structure (Westoby et al., 2012). Details on the SfM algorithm can be found in several publications (Agisoft, 2018;Kersten, 2016;Lowe, 2004). The primary product of the reconstruction is 3D point cloud, the secondary product is a two-dimensional orthomosaic (Gómez-Candón et al., 2014). Georeferencing (UTM zone 32N) of the point clouds was based on six ground control points (GCPs). For extracting the canopy height model (CHM) the 3D point cloud has to be subtracted from a ground model. The result enables the assessment of canopy height as illustrated in Figure 4. The CHM was rasterized with a spatial resolution of 0.01m. For further calculation the maximal height value for each grid cell was exported.

Lodging Percentage
The methodology was focused on a mathematical approach for lodging percentage assessment, avoiding adjusted thresholds and subjective decisions. As a first step, the maximum canopy height (MAXCH) of each genotype were calculated. Related to the MAXCH, three different lodging percentage threshold (LPT) were used to calculate the lodging percentage; 80% (80LPT), 70% (70LPT), 60% (60LPT). Finally, the lodging percentage were determined 75 days after sowing (DAS) by a query (rasterized CHM < LPT) resulting in a binary image with areas influenced or not influenced by lodging.

Lodging Development
The approach additionally enables the determination of recovery rate of crops and evaluates the influence of different lodging events based on a multi-temporal consideration of lodging percentage. The average lodging percentage was calculated for genotypes within experimental setup (

Lodging Severity
For the second lodging parameter four thresholds related to the MAXCH varied from 80 % (80LPT) to 50 % (50LPT) were used to calculate the average lodging severity (ALS) according to Equation (1). Additionally, the weighted average lodging severity (WALS) was calculated (Equal 2). In comparison to ALS the parameter WALS additionally weighted areas of the canopy affected by lodging differentiated regarding the yield impairment. The value range for both formulas varied between 0 and 100 %.

Lodging Validation
The area of lodging were manually determined in additionally acquired high-resolution orthomosaic (GSD = 2.3 mm, 75 DAS). Due to the very high resolution, the lodging area were easily identified.

Lodging Percentage
In order to identify an ideal threshold for UAV lodging percentage assessment, three different LPTs (80LPT, 70LPT, 60LPT) were compared to the reference measurement. The UAV lodging percentage derived from 80LPT let to the lowest correlation (R² = 0.892) in this comparison (Figure 5a). It became clear that the canopy height deviation between MAXCH and 80LPT were too small for most of the genotypes. Thus, the natural occurring canopy height variation was higher than the predefined threshold and lower grown canopy areas were partly defined as lodge areas resulting in an overestimation of lodging ( Figure 5a). The UAV lodging percentage derive from the 70LPT had a high correlation (R² = 0.96) and the least root-meansquared error (RMSE) (Figure 5b). The 70LPT considered the aforementioned canopy height variation in the field which resulted in high correlation and a low amount of scattering.
Comparing the reference measurement with the third UAV lodging percentage derived from the 60LPT, the correlation (R²  (Figure 5c). Canopy areas affected by lodging were partly not considered through the lower canopy height threshold. Thus, the lodging percentage was underestimated, especially in the less affected lodging plots, where the canopy height threshold was more relevant.
The UAV lodging percentage parameter was determined with very high accuracy in breeding trials (R² = 0.96). The results showed, that the 70LPT enabled the detection of lodged areas within the canopy and took into account the CH variance in the field as well. Liu et al. (2018) and Yang et al. (2017) reached R² greater than 0.9 and a high accuracy by assessing the lodging percentage in rice using structure, texture or thermal difference between presence and absence of lodging. But the accuracy strongly depended on the trained support vector machine (SVM) and the currently dataset. The quantification of lodging from thermal images is also very challenging, because external factors such as small changes in wind speed and cloud cover strongly influence the derived canopy surface temperatures (Chapman et al., 2014;Jackson et al., 1983). The lodging assessment through the UAV canopy height was much more independent from abiotic and external factors. Only a simple RGB camera was necessary without demand for calibration. In general, only large canopy height variations within a field can cause problems. In this extreme case, lower grown plants would be labelled as lodged plants. This issue, however, can be considered in the workflow by applying differentiated MAXCH values in areas with strong CH variations caused by different soil or nutrition conditions.

Lodging Development
The approach was additionally used to determine the lodging development like the lodging recovery rate or evaluate the influence of different lodging events based on a multi-temporal consideration of lodging percentage. Considering the multitemporal lodging percentage development for genotype 3 ( Figure  6), a low average lodging percentage value of 27 % was observed for plots with low sowing density. That indicated, that most of the plots were not affected by lodging at the beginning of observation (75 DAS). However, caused by a second lodging event, the average lodging percentage increased from 27 % to 60 % (81 DAS). The plots with high sowing densities showed distinctly larger areas that were heavily affected by lodging.
Regarding the lodging development, the average lodging percentage decreased from 80 % to 70 % at the end of observation. Figure 6. Average lodging percentage (%) with standard deviation for both sowing densities (high = green, low = orange) and genotype 1 (left). RGB image to illustrate the lodging pattern for genotype 1 (right).
The lodging pattern for genotype 1 was very similar among the plots. The center area was heavily affected by lodging, so that the main part of the plot completely lay on the ground (Figure 6). The outer area were mainly uninfluenced by lodging.
Considering the multi-temporal lodging percentage development of genotype 2, the aforementioned second lodging event could not be observed. Moreover, the lodging percentage for the plots with low sowing densities continuously decreased from 70 % to 55 % (Figure 7). In comparison to the other genotypes, genotype 2 nonetheless had the highest lodging percentage values in both sowing densities. In addition, the standard deviations for plots with high sowing density were very small. In contrast, the lodging percentage for plots with low sowing densities varied between 40 % and 90 %. Figure 7. Average lodging percentage (%) with standard deviation for both sowing densities (high = green, low = orange) and genotype 2 (left). RGB image to illustrate the lodging pattern for genotype 2 (right).
The lodging pattern for genotype 2 was also very homogenous.
Although the lodging percentage was in general very high, the lodging severity was low. Mainly, plant apices (spikes) were pressed down, without strong influence on the stems. The last genotype 3, had the lowest average lodging percentage compared to other genotypes, but the highest standard deviations. The average lodging percentage was again higher in the low sowing density compared to the high sowing density. The multitemporal observation indicated increasing lodging percentage, although no further lodging event was observed in the field. Figure 8. Average lodging percentage (%) with standard deviation for both sowing densities (high = green, low = orange) and genotype 3 (left). RGB image to illustrate the lodging pattern for genotype 3 (right).
The lodging pattern was quite similar to genotype 2 Only plant apices (spikes) were pressed down, without strong influence on stems.
The results showed that the average lodging percentage of the low sowing density was at least 20 % higher compared to the high sowing density. Already studies from Berry et al. (2002) or Berry et al. (2004) indicated, that lodging risk is reduced with lower sowing densities. This study confirmed additionally, that plants growing in the edges of breeder plots ( Figure 6) or plants growing near the wheel tracks were less prone to lodge than plants growing elsewhere in the field. Already the research from Scott et al. (2005) showed that the stronger resistance to lodging was caused by a higher stem strength of edge row plants resulting from reduced competition for resources. Finally, the multitemporal observation illustrated, that different lodging events can be monitored by assessing the lodging percentage at different time points (Figure 6). In contrast, the recovery assessment using multi-temporal lodging percentage calculation was more complicated. The lodging development can be influence by plants which sprout out again (Figure 9) due to the high lodging severity and early development stage of occurrence. Consequently, new grown plants (green) decreased the lodging percentage ( Figure 6, Genotype 1), but will not mature till harvest and influenced the yield quality negatively. Figure 9. New grown plants (green) after high lodging severity and early development stage of occurrence.
Through the natural seasonal development of cereal crops the canopy height was additionally decreasing from flowering (75 DAS) to ripening (102 DAS). Thus, the absolute height difference between lodge plants and healthy plants was decreasing, resulting in an uncertainty of lodging percentage calculation ( Figure 8, Genotype 3). For high accuracy, the lodging percentage should be determined at least two weeks after occurrence.

Lodging Severity
How already illustrated in the previous chapter, plants can be affected by differentiated lodging severities. Respective to the lodging percentage parameter the amount of affected plants below 50LPT were rated equal compared to plants, which were only slightly affected by lodging (area between 80LPT to 60LPT). The lodging severity approach with different thresholds (Figure 10) was able to consider the canopy height variation and the possible yield impairment caused by lodging. The 50LPT applied to the lower density plots allowed detection of only 35% of lodged area at maximum and 10% at minimum (Table 2). Contrarily, 70LPT determined a distinctly higher amount of 71% lodge area at maximum and 27% at minimum. The applied weighting procedure within the WALS calculation based on the different thresholds was able to consider this lodging intensity variation and, compared to the lodging percentage derived from 70LPT, led to a difference of 16% at maximum ( Table 2). The plots with high sowing density showed distinctly larger areas heavily affected by lodging, with 69% at maximum and 50% at minimum for 50LPT. Nevertheless, the variations still present between the different LPTs resulted in a deviation of 15% at maximum between 70LPT and WALS for the plots with high sowing density. Comparing ALS and WALS clarifies that the weighting procedure applied during WALS calculation consider additionally the yield impairment to a greater extent while ALS probably slightly overestimate the lodging severity. The maximal difference of the WALS values and the ALS values was 6 %. The lodging severity parameter WALS and ALS were able to consider the CH variance and the information density was compared to a simple binary approach much higher. Several papers (Bendig, 2015;Chapman et al., 2014;Liu et al., 2018;Yang et al., 2017) implemented only a presence or absence of crop lodging and the different lodging severities illustrated in Figure 10 were treated equally. The weighted method implemented to WALS parameter could improve the yield impairment caused by lodging. Already Fischer and Stapper (1987) or Berry and Spink (2012) showed that the yield potential was influenced by the intensity (angle) of the permanent displace from its upright position. Related to the UAV application also Murakami et al. (2012) showed, that the grain yield was impaired stronger by high lodging scores and a low average canopy height. The WALS development was the first step to predict the yield losses of lodge fields. The different factors applied in Equation (2) probably should weighted stronger the lower LTP values but has to be evaluate with yield data comparison in further studies.

CONCLUSION
Unmanned Aerial Vehicles (UAVs) are increasingly used, and open new opportunities, in agriculture and phenotyping because of the flexible data acquisition. It can provide plant breeders, insurance companies and farmers timely detailed information on plant traits with low monetary costs. Especially breeding trials are difficult and extensive to monitor resulting in an increasing need for a faster selection of superior lines. The lodging quantification based on the 3D canopy structure is compared to other approaches much more independent from environmental conditions, which strongly increases the practicability. Additionally, it enables the possibility to consider the yield impairment caused by lodging. The results showed that the developed method is well suited for barley genotypes and therefore has the potential to be applied to other cereal crops, such as wheat. The pixel-based lodging severity information can be further used in precision farming to generate harvest maps and improve yield quality by avoiding areas in the harvest process that sprout again after heavy lodging events during the early stages of plant development. In summary, the developed lodging assessment approach can be used for insurance applications, precision farming, and breeding research. This trait, together with differentiated recovery are novel traits next to lodging severity will aid the selection for genetic lines.