DEVELOPING A LAND COVER CLASSIFICATION OF SALT MARSHES USING UAS TIME-SERIES IMAGERY AND AN OPEN SOURCE WORKFLOW

Salt marsh ecology classification is difficult using traditional coarse resolution remote sensing techniques. Salt marshes exhibit a spatial pattern of vegetation zonation that are visually identifiable using imagery that has an improved 0.04 meter per pixel resolution. This project applies high resolution unmanned aerial system (UAS) imagery to aid in multi-temporal classification of our study area (Horseneck Beach) in Westport, Massachusetts, USA. We flew a DJI Phantom Pro 3 at lowand high-tide to capture effects the changing tide has on vegetation in an effort to predict effects of the rising sea level on saline plant species. We implement an open source software workflow using OpenDroneMap and the Semi-Automatic Classification Plugin for QGIS to create the necessary orthomosaics and to conduct vegetation classification required of this project. We compare land cover classifications using one-time-point RGB imagery to a multi-time-point (low tide, high tide) RGB image stack to investigate whether the multi-time point stack improves land cover classification accuracy. We find it does. More generally, this paper provides a model for others wishing to use low-cost UAS equipment carrying a simple low-cost RGB camera, and free and open source for geospatial (FOSS4G) tools, to develop multi-band image stacks to improve land cover classification accuracy. Further, we provide some reflections and technical notes on our experience. The approach we present here could be extended to include other image layers that UAS can provide when equipped with other sensors, such as multi-spectral (e.g., NIR, thermal), or by adding another band with photogrammetry-produced digital elevation data.


INTRODUCTION
The health of Massachusetts coastal marshes has been negatively impacted by decades of human intervention. Salt marshes provide habitat for rare and endangered species, prevent erosion and protect coastal areas, sequester carbon, and act as nurseries for fish and shellfish. However, projected sea level rise as a result of climate change (Melillo et al. 2014;IPCC, 2014) will cause marshland to become submerged or move inland, resulting in habitat changes or loss for fish and wildlife (Mitchell et al., 2014). For these reasons, we need improved methods for assessing these changes to inform management and public policy.
Land cover studies often utilize traditional remote sensing data products such as Landsat imagery (Jin et al., 2013) to map land cover, and the use of multi-temporal imagery has been shown to improve classification accuracy (Yuan et al., 2005). However, the spatial resolution of these traditional satellite-based data sources (e.g., 15x15 meters for panchromatic Landsat ETM+), coupled with the inability to acquire these imagery "on demand" limit the land cover detail they can provide. Mitchell et al. (2014) recognize the utility of higher spatial and temporal resolution imagery for more fine-scaled classifications of these landscapes, but raise the concern of costs over benefits. However, their discussion alludes to the use of more traditional remote sensing technologies (e.g., helicopters); the use of UAS-based remote sensing might dramatically reduce this cost versus benefit equation.
Consequently, the potential promise of UAS for higher resolution detection of salt marsh change has brought together scientists at the University of Massachusetts, Amherst and the Massachusetts Office of Coastal Zone Management to investigate its added utility. Specifically, we ask: Does a multi-temporal stack (low tide & high tide) of simple true color (RGB) imagery using a low-cost UAS improve the classification accuracy of an image orthomosaic compared to a classification of a 1-time (low tide OR high tide) orthomosaic created using the same UAS equipment?
In the study described below, we collected over fifteenhundred true-color aerial photographs for two time points in one day (a low tide period and a high tide period) using a DJI Phantom Pro 3 quadcopter. We then created orthomosaics and 3D models with the open source software OpenDroneMap and WebODM. WebODM operates as the browser based graphic user interface (GUI) for the UAS raster image processing Python and OpenSFM toolkit OpenDroneMap.
Using QGIS and the QGIS Semi-Automatic Classification Plugin (Congedo, 2016) and differentially corrected salt marsh field data collected by the Massachusetts Office of Coastal Zone Management, we ran a supervised classification of salt marsh vegetation first using separate one-time point RGB stacks (low tide and high tide), then using a two-time point RGB stack (low tide and high tide). We compare these methods to evaluate (1) how well low-cost UAS can assist in salt marsh land cover inventory, and (2) whether the low tide and high tide RGB stack improves landscape classification. Our goal is to improve the ability of halophyte mapping to discriminate between vegetation types throughout inner marsh and along natural and manmade channels using high resolution RGB imagery acquired by low cost UAS and open source image processing.

STUDY AREA
Our study area is the coastal marsh at Horseneck Beach in Westport, Massachusetts, USA ( Figure 1). The flight area encompasses one-hundred and forty acres of saline marsh managed by the Massachusetts Office of Coastal Zone Management. The dominant vegetation in the low marsh portion of our study area consists of Cordgrass (Spartina alterniflora).
Cordgrass are a mid-sized plant and grow along the creeks and ditches in the areas where it is more likely to flood, varying in height from 60 cm to 1.8 m (USDA, NRCS 2000). Spartina alterniflora can prevent erosion along shorelines, banks, and other areas where soil interfaces with water. The salience of cordgrass in the biotic community will increase as shore erosion becomes more prevalent. Capturing high resolution imagery of plants like cordgrass is an imminent necessity for spatial analysts looking to predict potential changes to wetland type due to rising sea levels.

METHODS
A FOSS4G UAS image processing workflow was used for spatial analysis and to create the models and band composites for this project. To create a geographically accurate image mosaic from hundreds of UAS-acquired aerial images, we utilized the ortho-mosaicing software WebODM. The resulting output are two one time-point orthomosaics and 3D models with a 0.04 meter pixel resolution of the study area.
Prior to the flight, differentially corrected ground control points (GCP) were measured to increase the spatial accuracy of the resultant orthomosaic for classification and stacking of the two time-points. We classified the image mosaics based on the RGB spectral signature of the vegetation with the Semi-Automatic Classification Plugin (SCP) for QGIS. The methods described here provide a FOSS4G image acquisition and processing workflow specifically for UAS imagery.

UAS System and Flight Preparation
We utilized a DJI Phantom 3 Professional quadcopter with a ½.3 CMOS sensor and a f/2.8 lens. Flight planning was done using the cloud-based system DroneDeploy.com.  A series of five flights were conducted on the 24th of August 2017, such that flight times were distributed evenly over a six hour period that coincided with a waning tide. The first flight launched at 10:50 am and the flight time was approximately eleven minutes. The flight altitude was 117 m and the flight path was chosen such that there was 70% sidelap and 70% frontlap between images. This lower level of overlap was selected to minimize the time between when the first image of each flight was taken and the last image was taken of each flight, thereby ensuring minimal change in water elevation over that time period. The subsequent three flights were conducted with the same parameters. The final fifth flight was conducted at a lower 76 m altitude and with 80% sidelap and 80% frontlap so as to achieve a higher resolution for a more accurate 3-dimensional terrain reconstruction, and launched at 4:39 pm ( Figure 3).
The time period between the first and last images captured was 71 minutes, and as such the UAS required four separate batteries and four distinct launches times over that time period. Given that the tide was at its lowest point during this period we expect that the water elevation change was minimal during this time. In the morning the weather was fair with only approximately 10% cloud cover, with cloud covering increasing gradually to mostly cloudy by the last flight.

Ground Control Points
Ground control points (GCP) were taken using a Trimble Geo7X provided by the GIS & Spatial Analysis Lab at Smith College. A total of eleven ground control points were measured in a circular pattern around the perimeter of the study area. Visual GCP location markers (neon cones on stakes) were placed prior to the flight at the location of the GCP in order to be identified in the raw imagery in image post processing.

Field Sampling Data
A networked Leica Viva GS15 RTK GNSS receiver with Leica controller was used to record Northings, Eastings, and Elevations for the purposes of collecting preliminary data on plant community composition at three CZM sentinel sites: Essex Bay (Essex), Barnstable Great Marsh (Barnstable), and Horseneck Beach State Reservation (Westport). The GNSS receiver was connected to MassDOT's CORS real-time network for real-time position corrections.
Data collection was opportunistic and primarily occurred in the areas adjacent to CZM sentinel site transects. The data collector targeted plant community patches onsite, being mindful of the number of points collected for each type of plant community. Sampling did not follow a sampling plan and all plant community patches were identified and characterized in the field.
A 1.5 m radius circle plot was used at each point (xyz position collected at center of plot). Plots were located well within a plant community or land cover type patch when possible to minimize spatial errors in the georeferenced imagery to be used for image classification. A meter stick was rotated around the plot center point (i.e., the RTK GNSS rover pole) to rapidly estimate the circular plot boundary. A visual estimate of percent cover for up to four plant species/morphotypes in each plot was recorded using the GNSS controller. The meter stick was then used to measure and record average height estimates for the two most dominant plant species/morphotypes, if applicable.

Percent Cover Cover Class ID
Trace 0 Percent cover class and heights were recorded for each plot with the GNSS controller using a string of alpha and numeric characters. Strings were 16 characters in length and used the following syntax. Each plant species/morphotype was given a two-character code, and when present, was recorded as such followed by a cover class ID (e.g., Spartina patens with 35% cover would be recorded as "sp2"). The four dominant species/morphotypes were recorded with their respective cover class IDs. Character placeholders were used where fewer than four species were present (e.g., "xxx", with the first two characters for species code and the third character for cover class ID

Image Orthomosaic Construction with WebODM
OpenDroneMap is an open source toolkit for processing aerial imagery. As described on the toolkit's GitHub, "OpenDroneMap turns simple images into three dimensional geographic data that can be used in combination with other geographic data sets" (OpenDroneMap Development Team, 2018). OpenDroneMap creates products such as point clouds, digital surface models, textured digital surface models, orthorectified imagery, and digital elevation models (OpenDroneMap Development Team, 2018). There are three main components to the OpenDroneMap project: (1) OpenDroneMap; (2) node-OpenDroneMap; and (3) WebODM. For the purposes of this project, WebODM was chosen as the appropriate component for its accessibility and added features. WebODM is the most accessible for users that do not have experience with command line functions or with advanced photogrammetry software.
WebODM is able to create georeferenced orthomosaics from EXIF data embedded in UAS imagery. To correct the positional error from the onboard GPS system of the UAS, GCPs are incorporated into the post-processing of the raw imagery. GCPs are processed by creating a text file that has several parameters including the Northing, Easting, Elevation, Pixel X, and Pixel Y for a given raw image and a GCP. A GCP text file consists of a minimum of five GCP and three raw images that correspond to that ground control point. The Northing, Easting, and Elevation are captured when the GCP is measured. Pixel X and Pixel Y refer to the grid value of the raw image that corresponds to the visual marker for that point. The .JPEG containing the visual marker is loaded into a seperate QGIS document and the "Identify" tool is used to locate and identify the center of the field marker.
The task of figuring out which GCP neon cone marker corresponded to which GCP location was an initial challenge due to the large number of images and lack of contextual markers (e.g., visible non-changing landscape features) at the study site. In their pre-processed format, the images do not correspond to a coordinate system. Thus their geographic orientation differs than when viewed as an orthomosaic, and adds an increased level of difficulty to the process.
We accomplish matching the GCP to the raw image by overlaying the GCP with uncorrected orthomosaic batches. The raw imagery was divided into batches consisting of fifty images and processed into spatially uncorrected orthomosaics. These batches were overlaid based on the positional data from the onboard GPS system and used to Four more GCP must be identified in the same method as described above for a minimum of 15 lines (5 GCP and 3 images per point). This is a time consuming process that required much thought about the most efficient way to match raw images to their corresponding GCP. We found that this is the most efficient way to complete the task for imagery that do not contain contextual markers. A total of 300 raw images were used for the high tide orthomosaic reconstruction and 500 for the low tide reconstruction. The differing flight plans between the two time-points produced more images due to the increased overlap and lower altitude.
To process this imagery, we used an Intel i-5 processor, with 2 TB hard drive, and 32 GB of RAM. The tidal imagery was processed at the highest settings capable of the computer without crashing the program and running out of memory. The pre-set setting "High Quality" was used with minor modifications. The "mesh-solver-divide" and "mesh-octreedepth" were increased to twelve. These options were chosen to increase the sharpness and visibility of the halophyte vegetation zonation. The processing of each tidal orthomosaic took three hours to complete with these components and the resultant spatial accuracy of the alignment is within 0.15 meters for the time-series The high tide and low tide orthomosaics differ on initial review due to different collection times during the day and different interactions between water level and vegetation due to lower sea level. These differences of depth and sun reflection are notable for future classification efforts, especially considering our stacked image analysis.

Data Preparation
We clipped a portion of both orthomosaics using QGIS from the 140 acre orthomosaic to limit the study area, by tracing a polygon around the perimeter of the low tide orthomosaic. The orthomosaics were clipped to the perimeter of the polygon to create matching image extents and flight paths. This was done to reduce error in the classification but also due to the varying flight plans when the imagery was acquired. The high tide orthomosaic captures a larger area at a higher altitude compared to the low tide orthomosaic.
The field data were split evenly between training and validation points. Thirty training points and twenty-nine validation points were used in the classification training. A circular buffer of 1.5 meters was created around the training and validation points to emulate the in-situ field data collection method.

Geographic Stacking of Low and High Tide Mosaics
The multi-band high and low tide orthomosaics were split into 8 single bands using the "Split Raster" tool in SCP. The single bands (excluding the Alpha band) were stacked into a composite six single band raster using the "Band Set Definition" and "Stack Band" tools. The Alpha band was excluded from the low and high tide mosaic stack as it does not contain spectral data. The resultant stacked raster composite will be used in the temporal classification.

Image Classification using Semi-Automatic Classification Plugin
We imported the buffered training data containing the CZM classification scheme to calculate the spectral signature for the 1.5 meter radius of the training points. Additional region of interest (ROI) training polygons were drawn to classify the water and solar glare into a single class, as it was not included in the field data collected by CZM. Our three classes, Low Marsh, Transitional Marsh, and Salt Shrub, were designated 1-3 and used as our MacroClass ID (MC_ID) for the classification. The additional plant species data was included as the Class ID (C_ID) for further specification of plant type for the given 1.5 meter radius of the field data sites.
We classified the separate tide images as well as the stacked image using Spectral Angle Mapping in SCP. The Spectral Angle Mapping calculates the spectral angle between spectral signatures of image pixels and training spectral signatures.
The spectral angle θ is defined as (Kruse et al., 1993): (1) Where: x = spectral signature vector of an image pixel; y = spectral signature vector of a training area; n = number of image bands. Therefore a pixel belongs to the class having the lowest angle, that is: (2) Where: Ck = land cover class k; yk = spectral signature of class k; yj = spectral signature of class j.

Figure 8. Stacked Band Composite Classification
We repeated this same process using the subset of validation points and the same water ROI training polygons on the low, high, and stacked tide orthomosaics. We used the accuracy The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4/W8, 2018 FOSS4G 2018 -Academic Track, 29-31 August 2018, Dar es Salaam, Tanzania assessment tool within SCP to compare these two classifications of training and validation points, creating a confusion matrix for each classified image.

RESULTS AND DISCUSSION
We successfully used UAS imagery and spectral classification techniques to classify three marsh types, and water, along the Massachusetts coast. The stacked RGB composite improved the classification accuracy in comparison to the single time-point classifications by 15% and 10%.

Time-Point Overall Accuracy
Low Tide 65.732%

70.523%
Stack 79.183% As the confusion matrix for the stacked analysis indicates (Table 4), the stacked classification did the best on transitional marsh (78.77% accuracy), but struggled with classifying the low marsh (8.81% accuracy). This could be due to the larger number of transitional marsh training points in comparison to the number of low marsh training points.
Comparing low tide, high tide, and the stacked classifications, the stacked classification is more successful in classifying the salt shrub class. For example, in the western portion of the orthomosaic the salt shrub class is more correctly identified when compared to the low and high tide classifications.
This is a promising start for future UAS classification efforts and demonstrates the potential for capturing "on demand" temporal datasets to aid landcover change detection and predictive modeling. However, there is still need for improvement and there are several flaws recognized in the methodology above.
Numerous technical challenges arose in initializing and actualizing this workflow. The installation process of WebODM on non-Unix machines is tedious and challenging for most users unfamiliar with command line functions. The WebODM component was chosen for its usability and accessibility but was an obstacle for most. The OpenDroneMap Development Team does have an installation wizard available for Windows users at a low cost of $25 that helps support the project. Additionally, the hardware components required to process the imagery were chosen for their accessibility but could increase the cost of this opensource workflow. However, overcoming this obstacle was crucial to the project's goal of democratizing UAS image acquisition and processing.
Second, issues on importing and processing the collected UAS data frustrated initial analytics. Principal of these issues is correctly aligning and overlaying each time-point in order to conduct an accurate classification. To overcome this, we created a workflow that is able to identify the raw UAS imagery with the appropriate GCP. Since this project was started, updates have been made to the WebODM platform to streamline this process. A new GCP extension was included in a recent update that uses visually identifiable objects in satellite imagery to georeference UAS imagery. This update is a promising start to making this process less tedious, but does not aid in areas that do not have visually identifiable landscape characteristics that are consistent in both UAS and satellite imagery.
Third, classification issues occurred, misclassifying the water as marsh vegetation occurs across classification time-points. These processing issues likely resulted from vegetation in the water and further exposed vegetation as the tide waned, which impacted analysis by distorting the classification of our main targets (Low and Transitional Marsh). Additional spectral information from added sensors (NIR) onboard UAS could alleviate this problem in the future.
Finally, calibration of the UAS imagery is an essential component of the photogrammetry process in order to reduce lens distortion (OpenDroneMap Development Team, 2018). However, this was not included in this workflow. The images were calibrated using the provided Python script available from the OpenDroneMap documentation. However, this script caused further distortion between the two orthomosaic time-points. To preserve alignment, the uncorrected photos for lens distortion were used for image reconstruction as they provided orthomosaics with a more precise spatial resolution and stacked alignment.

CONCLUSION
In this paper we investigate the utility of low cost UAS with a true-color camera that stores land cover reflectance information in 0-255 RGB digital numbers rather than in at-surface reflection --and an open source data processing stack --to see how well salt marsh land cover classification is accomplished. We take advantage of the idea that the additional layers of the landscape could be acquired efficiently and provide additional "ecological information" about the landscape by flying first at low tide, and then again at high tide in or around the same day.
The two time point stack suggests significant improvement in classification suggesting that future improvements can be made to the data acquisition and image processing methods. The utility of high resolution RGB imagery is demonstrated in this paper. However, the availability of added sensors aboard UAS is becoming more ubiquitous and accessible.
This project demonstrates the utility of stacking multiple time points of low-cost RGB UAS acquired imagery in that it provides more information in dynamic landscape environments than single time points alone. The same idea could be applied to capture land cover that exhibits seasonal changes (such as spring -summer -fall image mosaics). Further, the approach we present here could be extended to include other image layers that UAS can provide when equipped with other sensors, such as multi-spectral (e.g., NIR, thermal), or by adding another band with photogrammetry-produced digital elevation data.
This project exemplifies that an open-source workflow is implementable for UAS image acquisition and processing. Almost all of the tools used in this project are open-source, excluding the flight planning software and GNSS receivers used for field data collection and ground control point measurements. However, the OpenDroneMap project is continuing to be developed. There are several updates planned for the future, with flight planning software listed as an imminent goal (OpenDroneMap Development Team, 2018). We expect this project to provide a foundation for open-source UAS image acquisition and processing for future improvement as sensor technology and classification techniques develop.

ACKNOWLEDGEMENTS
We would like to acknowledge Jon Caris, Director of the Spatial Analysis lab at Smith College. This project would not have been possible without his collaboration in the spirit of FOSS. Without an RTK enabled GNSS receiver we would not have been able to collect the necessary ground control points for orthomosaic reconstruction. We hope that others hoping to use UAS to capture low cost high resolution imagery will find others that share his willingness to collaborate and find innovative solutions to geospatial challenges.
Kate Fickas, PhD candidate at Oregon State University, provided a number of helpful insights on remote sensing classification techniques that supported our efforts in exploring a stacked RGB composite classification of two time-points.
This project was inspired by the efforts of researchers at Texas A&M. We sought to build upon and improve their aerial classification methods using helicopters by replacing this method with unmanned aerial systems in an effort to improve and reduce costs associated with coastal marsh classification techniques.