WATCHING GRASS GROW-A PILOT STUDY ON THE SUITABILITY OF PHOTOGRAMMETRIC TECHNIQUES FOR QUANTIFYING CHANGE IN ABOVEGROUND BIOMASS IN GRASSLAND EXPERIMENTS

Grassland ecology experiments in remote locations requiring quantitative analysis of the biomass in defined plots are becoming increasingly widespread, but are still limited by manual sampling methodologies. To provide a cost-effective automated solution for biomass determination, several photogrammetric techniques are examined to generate 3D point cloud representations of plots as a basis, to estimate aboveground biomass on grassland plots, which is a key ecosystem variable used in many experiments. Methods investigated include Structure from Motion (SfM) techniques for camera pose estimation with posterior dense matching as well as the usage of a Time of Flight (TOF) 3D camera, a laser light sheet triangulation system and a coded light projection system. In this context, plants of small scales (herbage) and medium scales are observed. In the first pilot study presented here, the best results are obtained by applying dense matching after SfM, ideal for integration into distributed experiment networks.


INTRODUCTION
Globally distributed experiment networks have recently emerged as a powerful tool for environmental research across spatial and temporal scales.One example is the Nutrient Network (NutNet).collaborative composed of over 40 replicated grassland plot experiment sites spread throughout six continents.Key elements in the success of such networks are the low-cost of experiment setup and the data consistency benefiting from identical treatments and sampling [Borer et al., 2014].A key ecosystem variable is the annual quantitative determination of aboveground biomass on defined grassland plots.The current standard for sampling aboveground biomass is by harvesting a portion of the plot, drying the harvest and weighing it -obviously a destructive method with potentially impacts on the plot.Visual estimation or non-destructive physical measurement methods by experts have also been applied, but visual estimation suffers from subjectivity and a lack of repeatability of measurements, and physical measurements require significant sampling effort.
Photogrammetry promises efficient techniques to provide detailed 3D representations of grassland plot vegetation as a basis for biomass estimation, and complementary to the standard destructive methods.Obviously, due to the filigree structure of grassland plot vegetation, 3D representations provided by any photogrammetric technique will never be perfectly complete, and the volume derived from a subsequent 3D surface model is not equal to biomass.Therefore, a quantitative relation between the ✝ Corresponding author volume derived from 3D representations of plot vegetation and the actual plot biomass has to be established by imaging and harvesting representative sample plots in different scales, which is not addressed in the paper.

RELATED WORK
The study is focused on validating photogrammetric solutions to determine aboveground biomass on remote experiments sites and for low-budget research.Prior work includes testing the potential of TOF devices, dual stereo cameras, and SfM with subsequent dense matching.An extensive research of 3D imaging systems for agricultural applications is given by [Vázquez-Arellano et al., 2016].
TOF devices have been explored for imaging individual plants and with some preliminary testing on grass canopies.An experiment with a Microsoft Kinect v1, which uses triangulation with pseudo-random pattern projection, unveiled difficulties imaging individual plants under direct sunlight, but showed promising results for estimating height in grassland canopies at dusk [Azzari et al., 2013].In contrast, an experiment with the Microsoft Kinect v2, based on a TOF measurement principle, found moderately strong correlation to ground truth plant height and biomass under direct sunlight in outdoor conditions [Andújar et al., 2016].This experiment also showed an overestimation of plant volume because only the top of the canopy was captured, a problem to be encountered with future use of SfM for biomass estimation as well.The affect of various environmental conditions on two TOF devices in outdoor conditions showed water vapor, light intensity, and dust induced variation by millimeters in measured distance to target objects approximately one meter away [Klose et al., 2009].Coded light projection system was previously used to construct a 3D representation of broad leaved pepper plants [Bellasio et al., 2012], however no similar studies on grasses could be found.Similarly, broad leaved plants were imaged with laser light sheet triangulation systems in agriculture applications [ Šeatović et al., 2009].
Use of binocular stereo photogrammetry has been explored in plant phenotyping to measure plant shoot structure, leaf angle distributions and canopy structure of single or monoculture broadleaved crops [Fiorani andSchurr, 2013,Biskup et al., 2007].Binocular stereo systems will likely fail for plants with thin branches and narrow sheets due to ambiguities in image matching [Maas and Kersten, 1997].
The application of SfM techniques is increasingly recognised as a powerful low-cost alternative to laser scanning techniques for the generation of 3D point representations.Recently, studies for biomass estimation applying SfM with subsequent dense matching show promising results using terrestrial images as well as aerial images captured by Unmanned Aerial Vehicles (UAVs) [Lussem and Bareth, 2017, Forsmoo et al., 2018, Cooper et al., 2017,Bedell et al., 2017,Brocks et al., 2016,Ota et al., 2015].Besides biomass, [Jay et al., 2015] show the successful usage to estimate the height and leaf area of broad-leaved agricultural crops in monocultures.

SURVEY
In a pilot study on the potential of non-contact photogrammetric techniques for generating 3D representations of grassland plot vegetation, four photogrammetric approaches were examined on small-and-medium scale target plots including a "smart" TOF 3D camera using Google Project Tango, a laser light sheet triangulation system, a coded light projection system and, finally, SfM techniques.Targets for surveying different photogrammetric approaches varied because of logistical constraints.
Time-of-Flight 3D camera With establishing Project Tango, Google provides a comprehensive high performance low-cost device with inbuilt TOF 3D camera planned to develop augmented reality (AR) applications.In this study, the target for the TOF was a small, heterogeneous grass of about 50 cm above ground imaged outdoors at midday using a modified version of the Tango application "Java Point Cloud".With regards to this, 3D point clouds can be captured in a supported depth range of 4 m.
Laser Light Sheet Triangulation Beside TOF 3D cameras, laser light sheet triangulation was used to scan a Petroselinum crispum, characterized by very dense and curly leaves.With the measurement device MicroScan 1 , a laser line is moved by hand over the object of interest while a camera observes deformations due to the surface structure.Thereby, the sensor is fixed on a calibrated mechanical arm for determining the sensor pose while moving the device and thus, to align the individual scan stripes.This approach allows to capture about 30,000 points per second with single point accuracy of about 0.1 mm regarding a measuring volume about 1.5 m.Coded Light Technique In this study, the coded light system AICON smartSCAN-HE2 is used to generate 3D point clouds of Rosmarinus officinalis in the lab providing a single point accuracy of about 45 µm compared to a measuring volume of 48 cm x 36 cm x 30 cm (figure 2).
Figure 2. Apply a coded light approach to acquire 3D data of the small plant Rosmarinus officinalis in the lab.
Structure-from-Motion In the last decade, SfM increasingly became an alternative to Terrestrial Laser Scanning (TLS) approaches to generate coloured 3D point clouds of almost anysized surfaces using simple consumer cameras.The flexible multi-view approach is quick and easy to apply to images, captured from different perspectives, with homologous image contents.After feature point detection and matching, the camera configuration can be estimated via bundle adjustment for posterior dense matching.
As a basis for applying SfM and dense matching in the pilot study, two plots of 2.2 x 2.2 m 2 were imaged from 60 camera positions moving a camera around the plots in a height of about 2.5 m.A few ground control points (GCPs) and four calibrated scale bars were placed into the scene as a basis for scaling and referencing of the 3D model (see figure 3).The imaged plots differ in species type and growth structure.The first dataset covers a plot primarily composed of low-rising Onobrychis viciifolia, a legume species; the second plot is primary composed of the forb species Daucus carota reaching a height of approximately 0.5 m.Both datasets were processed applying the commercial SfM software package Agisoft PhotoScan.

EVALUATION
Time-of-Flight 3D camera The application of the Google Tango TOF-device turned failed almost completely due to daylight and vegetation reflection properties.On the one hand, the surface of an adjacently road could easily be captured but on the other, the approach failed as soon as the tablet was pointed Laser Light Sheet Triangulation Laser light sheet triangulation requires a relatively large interactive scanning effort to reasonably cover a small grassland plot; even if single blades can be captured, complex structures (such as blossoms, tangled blades) turned out to be problematic (see figure 1(b)).Regarding the development of NutNet, a high degree of automation is required, which nearly excludes laser light sheet triangulation for 3D data acquisition.Beside this, once the system has to be installed and calibrated, the base has to be rigid during the measurements which is an issue in outdoor applications.Also, with costs of a single measurement systems of more than 10,000 USD, the approach is far from low-budget science.
Coded Light Technique The coded light projection system would basically promise a very high accuracy and degree of automation, but inherently suffers from the viewing angles, sometimes unavoidably projecting patterns from one side of a blade of grass and taking the camera image from the other side as can be seen in figure 4.Moreover, the entire system is very space-, timeand cost-intensive in construction, calibration and purchase.In this study, the system was installed in one hour, calibrated in two hours regarding the binocular cameras and projector in relation to the measuring volume and 3D data was captured in additional two hours.Thus, the technique is neither an option nor a perspective for NutNet but rather intended for high-precision indoor measurements, e.g.quality assurance in automotive.

Structure-from-Motion
The best results were obtained by applying SfM-techniques to determine camera intrinsic and extrinsic parameters as well as a sparse 3D point cloud being densified using posterior dense matching.PhotoScan delivered surprisingly good results (compare figure 5).Especially in the case of low-growing Onobrychis viciifolia, individual leaf structure and stems can be well recognised in the 3D point cloud (see figure 5(a)).During image data acquisition for the taller Daucus carota, wind arose, causing problems due to sequential image acquisition, which can only be solved by applying a synchronised multi-camera system.Thus, figure 5(b) shows noise that caused by moving plants.Nevertheless, dense corollas could be reconstructed.
Due to cloudy weather conditions, both image datasets are free from disturbing shadows impeding feature point detection and matching.
Compared to the scale bars, which are calibrated with precisions about 0.02 mm, a Root Mean Square Error (RMSE) of 0.08 mm and 0.10 mm could be achieved for the 3D point clouds of the plots of Onobrychis viciifolia and Daucus carota, respectively (with regards to the internal precision figures from Photoscan).
Regarding Daucus carota, the accuracies have to be examined critically because the scales lied in the same horizontal plane which might be not an issue for surface distribution of Onobrychis viciifolia but of the taller Daucus carota.

FUTURE WORK
The 3D plot representations as shown in figure 5 do not provide aboveground biomass directly.However, the volume derived from the 3D representations will be related to aboveground biomass and can thus be used as a biomass proxy.Due to effects of occlusion, this relation will usually be non-linear.Occlusions and other systematic effects will be correlated in multi-temporal photogrammetric 3D representations, thus largely compensating in difference calculations.Therefore, temporal changes of volume will perform as an even better proxy for biomass changes.
As a simple technique for deriving aboveground biomass estimates from photogrammetric 3D grassland plot representations, the 3D data may be interpolated to a regular grid, deriving a representative height for each grid cell by statistical analyses of the 3D points inside the cell.The grid representation is next compared to ground truth data obtained by harvesting a representative sample of plots in order to obtain a function to translate volume into biomass.In future work, this quantitative relation should be established between 3D plot representations and biomass estimates with replicated grassland samples containing a combination of plant growth forms (e.g., grasses and forbs).The aim of such an experiment will be to compare the accuracy of SfM-based photogrammetry to standard destructive methods, and also understand the effect of varying environmental conditions on SfM.

Figure 1 .
Figure 1.3D data acquisition of Petroselinum crispum using laser light sheet triangulation.

Figure 3 .
Figure 3. Camera configuration to generate a 3D representation of Onobrychis viciifolia using SfM and dense matching.at grassland.Beside reflection insufficiencies, multi-path effects due to leaves refraction can be expected resulting in noise in the depth values.

Figure 4 .
Figure 4. 3D representation of Rosmarinus officinalis using the coded light approach.