COASTAL MAPPING AND KITESURFING

: Collecting data on aquatic biodiversity is very challenging because of the difﬁculty to access underwater ecosystems. Over the years, ﬁeld surveys have become easier and cheaper with the development of low cost electronics. Commercial and recreational vessels, including sailboats, can now substantially complement expensive scientiﬁc surveys and arrays of observation buoys deployed across the world oceans (Pesant et al., 2015, Karsenti et al., 2011). Meanwhile, a large variety of marine animals such as birds, mammals, and ﬁsh have become data collection platforms for both biological and environmental parameters through the advent of archival tags. It becomes obvious that data collection in coastal and high seas will become more popular and that citizen will play a growing role in acquiring information on ocean dynamics (physical, chemical and biological parameters). However, currently, very few attempts have been made to use Human beings as observation platforms. In this paper we describe large datasets (more than 200,000 pictures) that have been recently collected along the coast of Mauritius by using popular and cheap platforms such as kite surf and Stand Up Paddle. We describe the characteristics of the data collected and showcase how they can be geolocated and used to complement remote sensing and mapping in order to drastically extend the current scope of ”old school” ﬁeldwork. We point out some of the main limitations encountered which need to be addressed to foster this citizen science approach such as data storage and transmission, deep learning to automate image recognition. The methods are all based on open source softwares. SourceFile,


INTRODUCTION
There is nothing new in trying to make use of sailboats or surfboards to collect data.However we have very few examples of up and running projects at this time.Even if the cost of sensors is really low, this is mainly due to funding issues for what regards platforms to carry the sensors (e.g.sailboats) and associated costs like satellite data transmission which is still very expensive in high seas.However 3D printers are changing our ability to customize platforms and the smart phones and related sensors like action camera (which became natively waterproof) can connect each other and coordinate a network of additional sensors.Tracking the location of objects is a critical research field on earth but, at this stage, human beings remain unexploited biological platforms on oceans.This is only starting with big vessels, AIS and initiatives like Global Fishing Watch (Kroodsma et al., 2018) .Everybody can not afford a sailboat like Tara but many already have both action camera and boards (surfboards, kitesurf, SUP. . . ) which could be easily equipped to become efficient platforms for coastal surveys.Action camera are not used for every single session since conditions do not worth it all the time.However, instead of collecting images or videos about the unbelievable performance of the riders, action cameras, if turned underwater, might instead help to collect data about aquatic ecosystems.By doing so, we believe that the data deluge can become real for multiple coastal ecosystems and bring a lot of benefits and services to the society.Long term survey of coral reefs in overcrowded spots can quickly become a reality.Since action cameras videos (most of the time uninteresting) are hosted for free online, data storage should not be an issue if data become really useful.
The main limitations at this stage remain in the small, but still significant, economic and personal investment to turn his board into a powerful citizen science machine.From an economic point of view, out of surfing gears and actions cameras, our work has been achieved for less than 50 euros by equipping both a kite surf and a SUP boards and the cost was only related to drilling a hole for the action camera to film underwater while riding.Actually this was more a personal decision to be made.Once retired, like people, old boards can not really be sold and often end their life as memories hanged to the walls.However they can get a new life and become much more useful than young fellows with a little bit of customization.Since environmental consciousness should not be the only motivation, we want to set up a system which deals with safety and will be able to communicate the location to rescue teams.For what regards possible uses of such data, we foresee basic interest in seeing underwater landscapes, the possibility to measure the impact of climate change (e.g.coral bleaching event (Hughes et al., 2018a, Hughes et al., 2018b) or natural disasters like hurricanes).

MATERIAL AND METHODS
In this section we present how we collected underwater and aerial pictures.We will reuse the standardized terminology for ocean observing systems to describe the platforms used to carry the sensors and the method used for the deployments of this material in order to collect our data.

Human beings: Citizen kite surfing to foster data collection for marine ecology
Since years, citizen science has been identified as an efficient way to increase the scale of fieldwork and data collection for ecology (Dickinson et al., 2010).Leisure and outdoor activities (eg hiking) are good ways to provide data for science through collaborative platforms (eg Open Street Map, Flicker. . .).Kite have been used since years for aerial photography (Bryson et al., 2013) and surfing is identified as an opportunity to monitor coastal ecosystems (Brewin et al., 2016).Smartfin project already propose fins to collect physical and chemical parameters while surfing (Bresnahan et al., 2017, Stern, 2016).
Figure 1.Citizen kite collecting underwater and aerial pictures while surfing above coral reefs (Mauritius) As shown in Figure 1, Kite surfing is thus a good opportunity to collect data by equipping both the kite and the board.

Hardware: Platforms
The platform is the gear which is used to carry the sensors (in our case the board or the kite).As shown in Figure 2d, different types of boards can be equiped (by drilling a hole) and the equipment of the kite itself is made by attaching the camera directly on the kite.
According to the type of board and the type of camera the shape of the box has to differ.The work is almost the same to equip a board for surf, paddle, windsurf or kite and cameras have to be set up in the middle of the board (not to close to the fins, see Figure 2b).However kite twin-tips are a bit different as they are symmetrical (see Figure 2c).So far, we used an handcraft waterproof box to package sensors.At this stage, we still need to develop waterproof boxes adapted to the different platforms.For better results, low cost and open access to all, such a work should be done in collaboration with Fablabs.In the case of Kite surf, the kite and the board can be equipped for the same survey.In the table 1, we compare the strength and weaknesses of the different platforms used so far.Note that in the case of Kite surf, the kite and the board can be equipped for the same survey.

Hardware: Sensors
So far we mainly used a GPS sensor along with an action camera.Some recent action cameras already have a GPS sensor within but most of them don't and, in any case, GPS radio signal don't reach very far underwater.This is the reason why we used additional systems to record the spatial tracks of the survey.A watch with a GPS has been used to ensure our ability to collect GPS positions but any smart phone can now collect similar data with free applications (eg GPS light).The GPS sensor was parameterized to collect one location every second.
At some point, additional sensors are expected (e.g.sea surface temperature, depth. . .).We expect to build a waterproof case (by using 3D printers) for sensors to be deployed on any platform.Indeed, at this stage, even if action camera are often waterproof, water is a strong limitation for GPS data transmission and to embed electronic cards and sensors.For now, we used the following sensors: • onboard / underwater • on the kite (lines of the kite are more than 20 meters high) to get aerial pictures along with underwater pictures a light weight action camera with an embedded GPS (eg Gopro session 5) which can be equipped with filters (e.g.polarized), future plan: multispectral, lidar.
Depending on areas, we believe that very high resolution maps (less than 1 cm) could be generated with a mosaic of aerial pictures.However, to achieve this, there is a need to manage accurately the height and angle of the camera since the kite is often going up and down (with an Inertial Measurement Unit, IMU).

Open Source Softwares
This project aims first at providing the hardware and softwares for anybody interested in collecting data while surfing.To achieve this, a set of open source codes has been created to deal with the collected data.All codes (mainly R) can be executed on a Linux OS: • As delivered by the GPS sensor, data are provided with proprietrary tcx format which can be turned into GPX data by using the cycleRtools package or online applications (eg gpsvisualizer.com).GPX format can then be then turned into any spatial data format using gdal library.
• gpscorrelate has been used to geolocate underwater pictures colleced by action cameras.
• metadata of deployments (sessions) have been edited with a simple CSV which can be uploaded and collaborativaly edited as a google spreadsheet.The google spreadsheet is then programmaticaly accessed from R (by using the gsheet package) and turned into OGC 19115 metadata standards for geographic information (by using the R geometa package (Blondel, 2017) to generate ISO 19115 metadata and geonapi to push metadata directly in a Geonetwork catalog).Such codes can be compiled online by using RStudio server.
• pictures metadata have been extracted by using the exifR package and are then stored in a PostgreSQL / Postgis database to improve data management (including tags of images to train Neural Networks).
• maps have been generated by using QGIS (QGIS Development Team, 2009).
All codes used to synchronize GPS and camera, rotate images and create metadata will be made available online in the coming months.

Methods / deployments
First tests have been performed on coastal areas with clear water in order to collect underwater pictures.Our primary goal was to test our method to collect data on coral reefs habitats.Data presented in this paper all come from lagoons and related external slopes of Mauritius.However, it is clear that similar data can be collected wherever water is clear and shallow enough and where people can practice similar activities (snorkeling, paddle, kayak. . . on rivers, fjords, abers, . . .).Data described in this paper, have been mainly collected by using a kite surf and a paddle but we tested the method by snorkeling and surfing.As explained in 2.2, kite surfing is particularly interesting to cover wide areas with shallow water and it enables to combine both aerial and underwater surveys.
We used two different kinds of action cameras with the snapshot mode to collect picture at a frequency of half a second.This way we expected 7,200 pictures per hour.For last deployments we set up cameras for both the board and the kite in order to synchronize aerial and underwater survey as shown in figure 2d.The action camera used for the kite directly embeds a GPS.

RESULTS
In this section we present some preliminary results.We will first describe some examples of deployments (surveys) and samples of related data (underwater and aerial pictures).

Surveys
So far, more than 30 deployments have been performed in different lagoons and more than 200,000 images have been collected.Since GPS and camera use different clocks, we had to correlate them to infer the geolocation of each picture (same issue happens with temperature observations).The Figure 3 showcase different kinds of tracks depending on platforms used (Kite, SUP, surf, snorkeling).
As expected, surf sessions cover very limited areas because they are focusing on the peak of waves and paddling is very slow.Paddle have the same characteristics as surfboards when surfing but most of people use SUP to go anywhere which makes SUP more intersting than surfboards to collect high quality data.However, they are usually as long as kite or paddle sessions which means that we will have some oversampling on these spots which will enable long term observation network to get time series, accurate 3D mapping. . .Kite surfing are way faster, can go anywhere but image quality is lower at full speed (a 2 hours survey can cover more than 50 km).The Figure 4 gives an overview of what can be achieved within a lagoon with multiple (16 in this case) sessions.
This gives an outlook of how citizen science could be used in some specific spots to get time series for coral reef monitoring.

Underwater pictures
Depending on the platforms (kite, paddle. . . ) the speed and thus image quality and covered areas are very different.The Figure 5 gives examples of underwater images (with a very good quality).
As human beings we tend to only consider good quality pictures: good light and visibility as well as other aesthetic criteria.However most of data collected when surfing are collected randomly since the rider don't know exactly when the picture will be taken.Indeed, when riding at 20 knots, images are taken every 5 meters if the snapshot mode is parameterized with a half a second frequency.
For what regards blurry images, it is important to remind that current image recognition algorithms can deal with noise and low quality data to recognize habitats.Convolutional Neural Networks have shown their ability in making use of low quality images.At this stage, we expect that even low quality images will become useful.
Depending on environmental conditions (sun, wind, rain, water depth) and depending on the platforms (kite, paddle. . . ) the speed, the surface of covered areas and thus image quality are very different.
For some deployments, we have been able to combine underwater pictures with aerial pictures taken from the kite.

Aerial pictures
At this stage, aerial pictures have been collected to showcase how the combination of underwater and aerial surveys can bring valuable results.Kite have been used since years for aerial photography (Bryson et al., 2013).However, kites used for surfing are not as stable since they are constantly moving up and down.
Since aerial images overlap, we could get a photo mozaic all along the survey.However, to achieve this, we need to set up an additional sensor to monitor the angle of the camera (IMU) and enable the orthorectification of highly heterogeneous images.
The Figure 6 shows examples of aerial survey and illustrates how they could be used to complement traditional aerial surveys or remote sensing data.Figure 6c shows two rays at the surface.

Related applications
We foresee different applications to exploit these data in our domain of interest.However it is clear that these data might be used in many other ways and should be shared with the community.

Image recognition with deep learning
Deep learning can be used to automate image processing and recognition (Beijbom et al., 2015).In our case such a technology is key to deal with the large amount of data which can be collected.
As a very first step, we checked that generic deep learning algorithms were able to recognize objects on our images (see Figure 7) Figure 7. Amazon Web Service: image recognition of a ray In order to train neural networks to recognize specific objects, it is required to annotate some images which contains these objects.
In addition, pictures used for training of neural networks have to be resized since the native size (3648*2736) is too big and should be not exceed 500*500.
A first part of the work is going to be run in our research unit (tags of pictures and CNN).Thereafter we plan to set up a kaggle competition (https://www.kaggle.com/competitions)or similar challenges (eg SEACLEF (Joly et al., 2017)) to improve the quality of image recognition.
Once image recognition will work, we will thousands of points with additional attributes indicating what kind of objects have been identified in related locations.

Mapping Coastal Areas
Images are geolocated and, for some of them, overlap.Depending on the quality of images, they might be used in different ways for mapping.

GIS and remote sensing
Our main goal was to map coral reef habitats by using collected pictures as fieldwork and observations to validate maps.Indeed most of the fieldwork has to be done with a boat and researchers snorkelong or diving.We will thus complement the traditionnal approach with additional pictures annotated by deep learnig algorithms (CNN).
Photogrammetry: 3D mapping of coral reefs From images which overlap (depending on speed and time lapse) or videos, it is possible to obtain 3D models with stereoscopic methods.Since images are geolocated, it becomes possible to estimate the depth as well.
Figure 8 shows an example of 3D mapping of a coral reef which has been made by using pictures collected from a paddle.
Figure 8. 3D mapping from coral reef images collected from a paddle In the coming months we will focus on collecting higher quality images to showcase how SUP can be used for 3D mapping of coral reefs.

Coastal Zone Management
More widely, such data (GPS tracks and pictures) can be useful to manage coastal zones by getting a better knowledge of areas of practice, avoid risks and potential conflict uses.This type of data is now key for Marine Spatial Planning.
However since such data might be reused in other ways, we plan to release them as open data in the coming months.

DATA MANAGEMENT
Since we aim at making the acquired data available for coastal and ocean observing systems, we promote the compliance with widely used standards for data management (in particular OGC standards).

Repositories for open data storage and access
Making these data open by uploading them on Zenodo (except potential issues with endangered sedentary species) has been identified as a good way to ensure long term data archiving, dissemination and citation (DOIs).Various repositories can be used for such research data: • Zenodo and pangea to store raw data • Kaggle will be used as another channel to disseminate resized pictures (since deep learning requires lighter images).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4/W8, 2018FOSS4G 2018-Academic Track, 29-31 August 2018, Dar es Salaam, Tanzania This contribution has been peer-reviewed.https://doi.org/10.5194/isprs-archives-XLII-4-W8-3-2018| © Authors 2018.CC BY 4.0 License.However, since sharing location of certain species can become an issue for biodiversity (e.g.fostering illegal fishing), we plan to set up an image recognition algorithms prior to any data sharing.However it might be possible to share images without sharing location in order to enable image recognition on photos.
Since publishing data on zenodo only requires a minimum set of metadata elements (Datacite standard), we decided to enrich these basic metadata elements to foster data discovery for the marine domain community.To achieve this, we chosed to comply with geographic metadata standards (OGC / ISO TC 211 standards).

Metadata and data access
Data discovery is driven by metadata.In addition to pictures metadata (exif), we decided to describe the series of images (meaning the sessions) in order to enable spatial and temporal queries.OGC metadata have been generated by using google spreadsheets and open source codes (for both R packages and scripts).R geometa package (Blondel, 2017) has been used to generate ISO 19115 metadata and geonapi to them in Geonetwork.Metadata from different deployments can be stored in a Postgres / Postgis database.
Usually the raw data record is a picture which might need to be resized or rotated (eg for image recognition through neural networks) and has to be geolocated.Once underwater pictures collected, they have thus been processed to correlate the clock of onboard action cameras with GPS data clock (by correlating the watch of the action camera with the one of the GPS, as described in section 2.4).This way, we infer the location of each picture, store the latitude and longitude in the headers of jpeg files and obtain geolocated pictures which can be displayed on a map (see Figure 9).A the end of the workflow, some of the exif metadata stored in the headers of JPG files have been extracted and stored with the deployment metadata in a PostgreSQL/Postgis database wich can be used to filter images and store tags (in particular labels to train neural networks).Amon possible metadata elements we extracted the following ones: SourceFile, FileName, Directory, GPSLatitude, GPSLongitude, GPSPosition, GPSDateTime, DateTimeOriginal, LightValue, ImageSize, Model.

CONCLUSION AND OUTLOOKS
This paper presents preliminary results of an ongoing project which aims at collecting data on aquatic ecosystems by using citizen contributions through sports like kite surfing, SUP, snorkeling.The suggested method showcases how thousands of images can be collected every day by one single rider.In addition, it is possible to acquire data on other environmental parameters (temperature, depth. . .).The first tests are promising for what regards the quality of pictures which can be geolocated by using external or embedded GPS and can be used by Neural Networks to automate image recognition.Indeed, at this early stage, we already have valuable pictures and we face data storage and data processing issues.Methods for data description, storage and processing are key in the context of citizen sciences.A workflow for data management is required to ensure that a collaborative approach will be sustainable and will enable to properly manage the life cycle of pictures and other observations which will be collected in the long term.A first workflow has been set up by using open source codes and softwares.However we need to find simple ways to get new citizen scientists onboard and we still face multiple challenges to equip platforms, simplify data transmission and customize sensors.One of the mshort term challenge consists in improving the accuracy of GPS location (using Real Time Kinematic, RTK) and in managing the angle of pictures taken from the kite (Inertial Measurement Unit, IMU).This will require some work to customize the camera with additional low cost components.In addition we plan to add some additional sensors (like fins from smartfin project to collect environmental parameters).Since we estimate that an active contributor can provide more than 1TB of pictures per year, such a project needs to facilitate data transmission as well as data processing and visualization.Links with external communities (eg OSM for spatial data sharing or Pl@ntnet) will be key to foster the growth of such a project.

Figure 2 .
Figure 2. Setting up cameras on boards or on the kite

Figure 3 .
Figure 3. Different types of survey trajectories according to the gears used

Figure 4 .
Figure 4. Location of all pictures collected in Le Morne lagoon so far (16 sessions: kite, snorkeling and paddle) Figure 5. Examples of underwater pictures

Figure 9 .
Figure 9. Correlation of pictures and GPS tracks