DEDICATED PAYLOADS FOR LOW ALTITUDE REMOTE SENSING IN NATURAL ENVIRONMENTS

The recent years have shown a growing interest in low-altitude remote sensing for the study of natural areas. But natural environments lead to many constraints on acquisition sensors, which add to operational and carriers constraints. This article is a feedback on the design of two of these sensors.


INTRODUCTION : A NEED FOR DEDICATED SENSORS
Observation of natural environments with remote sensing is now a common practice for biodiversity studies and land cover.More and more scientific teams employ low-altitude remote sensing (LARS) for such studies (Lu, 2006), (Jensen, 2007), (Berni et al., 2009), (Torres-Sánchez et al., 2014).It allows to perform a more precise cartography and to identify the species of small-sized individuals that were not observable at larger scale.
The most used sensors in these studies are imaging sensors.Most of the projects use off-the-shelf digital cameras without major adaptation to their carriers or thematic needs.
But beyond the problematic of access to the study environment, the mapping of natural areas on a large scale poses many constraints to these acquisition sensors.Indeed, the environments are difficult for traditional image processing algorithms: inherently moving (at observations scales of individuals), composed of hardly distinguishable objects, these environments present varying weather conditions, and sometimes even hostile conditions (lighting, humidity, temperature, wind, etc.).The mapping task is usually up in a thematic study which adds its own operational constraints (need for 3D information as relief or roughness, specific spectral signature, oblique views to simplify visual identification by experts, etc.).Furthermore, the use of low altitude remote sensing lightweight carriers, such as micro-UAVs, severely limits the available resources for sensors: embedded power calculation, size, weight, etc..
In consequence, the feedback of the thematic users shows a great need of innovation for dedicated sensors for environmental acquisition in LARS (Labbé, 2014).
In this article, we will present two sensors specifically dedicated to image acquisition in natural environment : a tri-cameras sensor and a binocular stereoscopic rig.The first part of this article concerns the operational constraints due to the environment, the end user or the acquisition system itself.Then, we deal with the developed payloads and detail our choices given the explained constraints regarding sensor types, control, synchronization, record, dating, orientation, etc..The last part shows some results obtained for monitoring biodiversity on a Natura 2000 area.

Environmental constraints
We will focus in this study on the transition areas between forest and moor as these areas have a quicker biodiversity dynamic and often require a special attention.At the LARS scale, these environments are very complex because of the huge diversity of the individual natural elements (fauna and flora) that composed the observed scene.The natural movements due to the wind for example are visible at this scale and are problematic for most image processing algorithms.Moreover, the quality of the images acquired depends of the natural lighting and visibility conditions.And those could be highly variable during the mission.Georeferencing of data is essential to register them in a global reference system, especially at this observation scale.LARS studies are performed close to the ground and thus the surrounding relief may prevent access to reliable and precise global location (GPS) like it is often the case in mountains or at the bottom of valleys.In these cases, alternatives must be found.
Finally, we can note that some applications like the study of phenological properties or the analysis of a complex natural system, the moment and/or the frequency of the acquisitions is very important (Vioix, 2004), (IFN, 2010).

End user constraints
Data acquired for scientific studies have specific constraints of their own.
For example, one of the main needs concerning identification of plant species in biodiversity monitoring is the creation of very high resolution orthophotos (with subcentimetric or millimetric ground sample distance) for mapping phytogeographic groups (Gademer, 2010), (Mobaied, 2011).These data could be completed by oblique images that allow estimations of the height of individuals as well as an overview of the study area.This information therefore gives a better understanding of spatial relationships between the different elements and allows a richer interpretation by specialists (Petrie, 2009).
The LARS also allows the introduction of new applications like dendrometric studies from images (physical measurements such as plant height or shape, distribution or population density, etc.) Figure 1: Top: image of small bushes taken from a fixed point of view.Bottom: the brightest areas show the biggest displacement of the vegetation during a period of one second.
or roughness surface extraction for the interpretation of radar signals.These applications are based on very accurate threedimensional models of the observed scenes (typically at centimetric scale), and so, constraint the way the data are acquired ((Petitpas, 2011)).The first return of experience from these examples is that it is particularly important to involve the data end-user in the payload design phase.This is a very sensitive point because both the technical integrator and the end user tend to imagine the specifications instead of the other one.So the difficulty is to define the real needs related to the thematic studies regardless the technological point of view that comes after.

Acquisition system constraints
The lightweights systems used by LARS severely limit the resources available for the sensors they carry: power, embedded computing, size, weight, etc..
The average payload that can be lifted by a micro-UAV is usually between 200g and 1kg.This limitation is not a really technical limitation but is imposed by legislation.Indeed, due to the exponential use of drones those past five years, most countries have adopted a particular regulation.Today, there is not international rules, so it is very important to know the local rules where the UAV is flying.In France for example, the total weight of the system (micro-UAV with its payload) should be under 2 kilograms (D category).Some drones of superior categories or some rc aircrafts can carry heavier payload but this implies much higher economic and legislative constraints (MEDDTL, 2012).
The sensors used on these carriers mainly consisted of digital cameras equipped with a radio-controlled trigger or a timer.If this solution may be satisfactory for producing orthophotos (once solved vibration and distortion problems of the camera), it is generally incompatible with stereo-reconstruction when acquiring data at small range.Indeed, sudden movements of flying micro-UAVs (due to the wind or changes of direction) could blur images during the acquisition.Furthermore, inherent movements of natural environments will introduce errors in the reconstruction because the images are taken at different moments.To give an idea on the quality impact on 3D centimetric reconstruction model of bushes vegetation for example, we have taken asynchronous and synchronous images from a fixed point of view under low wind conditions (less than 8 km/h).Figure 1 shows one image of the set and a localisation of the main vegetation displacements on a period of 1 second.
The longest and highest branches, which are also the more flexible, are the ones that move the most.The consequence is that matching algorithms are ineffective in these areas because the implicit stereoscopic hypothesis of observing a perfectly identical (static) scene is not respected.Figure 2 shows the reconstruction obtained using unsynchronized images and for comparison a reconstruction using synchronized images.As expected, the black areas where the reconstruction algorithm failed are located where there are the biggest displacements.The synchronization of the acquisitions of LARS images for 3D reconstruction is critical and thus implies the used of at least two cameras.

DEVELOPPED PAYLOADS
From the constraints outlined above, we have developed two payloads dedicated to the acquisition of LARS data in natural environments (figure 3).For these two payloads, we have been particularly attentive to the synchronization of acquisitions in order to be able to neglect the consequences of the natural dynamics of environments.In addition, we chose to use low cost materials, making these payloads easily replicable at a reduced cost and accessible to communities with limited financial resources.The first payload is dedicated to the study of biodiversity in the transition areas of forest and moor and was tested on a quadcopter UAV.It consists of three cameras which allows a high overlap between the simultaneous acquired images.The three cameras can be independantly oriented for shooting nadir or oblique views.
The second payload allows a perfectly synchronized acquisition of stereoscopic couples and can retrieve data in real time.Designed for aerial and underwater use, this payload is particularly well adapted for the study of coastal areas (Avanthey et al., 2013).

Sensors choice
For our first payload, we chose to use off-the-shelf compact digital camera (Pentax Optio A40) in order to combine minimum weight (120g when stripped of their cases) and internal storage of high quality 12 Mpix still images.As our quadcopter has a 1kg payload capacity, it allowed us the design of a three cameras system that will be able to get several point of views at the same time.
For our second payload, we preferred a sensor natively allowing much greater control and the ability to directly access to the images acquired during the mission.Our choice was therefore focused on small digital imaging sensors, with a good compromise on the sensitivity and resolution.The model finally chosen was a uEye LE IDS 1.3 megapixels, for a weight of 16g.That kind of sensors requires an embedded computer for control and storage.

Sensor control
The sensor control provides two remotely major actions: switch on/off sensors (it is critical to put them in a safe mode during the take off and landing phases) and simultaneous automatic trigger shooting.For the first payload, the Optio camera are designed to be handled directly by humans and not by an electronic system (triggering by push-buttons) Lot of projects use small servomotor to press these buttons.But this mechanical solution is sensitive to vibrations.Others use triggering functionality through the USB port available on some devices like the Canon or Nikon, but these systems equip mainly Single Lens Reflex (SLR) camera incompatible with our weight constraints.An other solution is to use the infrared sensor of the remote control (PRISM).Our tests with the infrared sensor of our camera has shown a long latency of this system that lower the possible frequency of shots (1 image every 3-4 seconds) and no guarantee that the image have been taken (the failure rate may be as high as 40 to 60 % !).It was clearly unusable for image synchronization.For this reason, we have chosen a last intrusive but effective solution : the hacking of both debugging signal and release buttons (power and trigger) of the camera through an homemade electronic system (Gademer et al., 2009).
We thus improved the datation of acquisition to 20ms and if the illumination conditions are favorable, the frequency between shots is reduced to one image every 1.5 seconds.As for synchronisation, this solution allows us to reach a rate of about 0.05 second.This synchronization rate is good but could be insufficient for some applications.The limits of that kind of hacking solutions is the internal firmware and processing algorithms of the camera (fig.4).This prevent to reach a high level synchronization.So, it is important to fully control the internal processes and it is the reason why we have developped an other payload.
Indeed, the uEye sensors are fully programmable (acquisition parameters, image processing time) and have a native triggering wire for triggering synchronization.Figure 6: Example of an oblique image acquired int the field area each camera in one thread, we reach a synchronization rate of about 0.005 second.The frequency of shooting is an image per second (limited by the transfer rate and the speed to write images to the disk).We periodically readjust the exposure settings to better adapt to the environmental conditions that could be highly variable and thus avoid over-or under-exposure.

Recording, dating and processing data
The first payload directly stores the acquisitions on its internal memory card.The recorded date is that of the camera and saved in the EXIF file.The precision is only at the second scale, which is clearly insufficient.To overcome this, we had to plug our electronic card on particular places in the camera to detect the specific debugging signals that are emitted when an image is actually taken.By listening to this signal, we can confirm shooting date and with an accuracy of 20 milliseconds according to the system time (CPU of the electronic control card).With this payload, the images can not be recovered and treated before the end of the mission.In the other hand the image resolution is much higher.
The uEye cameras do not have their own storage capacity.The images are transferred to the onboard computer and stored on a memory card.The dating is done directly by the onboard control electronics (CPU time, accurate to the thousandth of a second).
Once arrived on the onboard computer, the data can be processed on the fly (completeness of the data acquired for a live feed back on the progress of the mission, navigation, etc.) during the course of the mission, or analyzed at the end of it.

Sensor orientation
The orientation of the three cameras of the first payload are individually adjustable on demand on the pitch axis (45 degrees back or front or nadir).This gives the possibility of stereoscopic couples with at least two cameras or to make oblique acquisitions (45 degree front and rear) simultaneously with a nadir one.
On the second payload, the cameras are placed in a rigid geometry for stereo purposes.The whole sensor can be fix vertically or horizontally before the mission depending of the application.If  necessary, it will be possible to fix it on a pan-tilt system, but to avoid any calibration problems the rigidity between the cameras must be preserved.The distance between the two cameras (stereo baseline) can be adjusted according to the needs of the mission (working distance to the target, movement speed, required accuracy, etc.).

Weight constraints
When working with a flying machine, one should always consider the weight as a crucial factor.
For the first sensor, we wanted a multi-cameras system that weighted less than 500 grams to be carried by our customized quadcopter (the weight of most single digital SLR camera is about 1Kg).With this in mind, we selected light-weight compact cameras (Pentax A40, ∼ 150g) and we have stripped them down of their shell to reach ∼ 120g.For the frame, we started with a 3D printed plastic case (∼ 85g), but we finally choose a carbon case that was sturdier and lighter (∼ 26g !). Adding some servomotors and the home-made electronic control card, the total weight of the system was ∼ 450g for three adjustable cameras with trigger control and precise dating.
The two uEye cameras are already light-weight with only 35g, but they need an embedded computer in addition.To keep the

Erica cinerea
Rejet de pin  weight the lowest possible, we have chosen a linux operated android TV-key (UG802, ∼ 30g without its case).This computer have a dual-core 1.2Ghz processor that allow us to process both our uEye camera and store a stereo couple by second with lossless compression.A rigid frame and some cables complete the system.The total weight of the uEye stereo rig is ∼ 180g (and ∼ 460g with their underwater housing).

Calluna vulgaris
Table 1 shows the detail of the repartition of the weights.

QUALITATIVE ANALYSIS OF MONITORING BIODIVERSITY OF A TRANSITION AREA BETWEEN FOREST AND MOOR
In collaboration with a team of botanists of the National Museum of Natural History we have organized a field mission on their study site called the "Mare Joncs".The objective was to identify the potential contribution of LARS for their biodiversity monitoring studies.1800 pictures were taken with the tri-cameras system during the 5 hours of flight either in stereoscopic or oblique position.LARS allows a fine control over the fly height and such allow to take in the same day some global views of the field and very high resolution images on point of interest.Figure 6 shows an example of oblique images taken from 75 meter above ground.The acquired nadir images have a Ground Sample Distance (GSD) of about ∼ 1.5cm .
For the qualitative analysis of the data, we provide to the botanist team different georeferenced products computed from the data acquired by the tri-cam payload.These products were high resolution orthomosaics, unsupervised automatic classification using Maximum Likelihood Estimation algorithm (Data et al., 2008) and 3D reconstructions.
From the High Resolution orthomosaic (1.5cm GSD), the botanists made the classification manually (figure 10) and compare the results with their own last cartography of the site realized in-situ.
The qualitative analysis shows that they have been able to identify four different species on the seven ones really living on the area.The global land-cover obtained is identical on about 62% with ground measurements 92% of tree higher than 1 meter has been identified.The three species not identified were too small or too spatially diffuse in respect to the others.All species that have a significant spatial extension have been found and the identification of small individuals have been possible.Indeed, unlike conventional aerial images on which it is possible to distinguish the large families (heather, grasses, woodlands), these data has shown the possibility of identifying thin woody species (heather, moor grass, etc.) and of distinguishing even more discrete species (bell heather, sheep sorrel, etc.) by further reducing the pixel size on the ground (figure 9) (Mobaied, 2011).With an other set of images having a 0.4cm GSD, six species on seven and most of the suckers (¡ 10cm) have been identified.
Unsupervised automatic classification (figure 7) seems to give a good first approximation of the total surface occupied by different species that have a significant spatial extend, but this analysis has to be confirmed.
3D point clouds extract from the stereo pairs (figure 11) is fully coherent with ground mesurements.From the global point of view (∼ 1.5cm GSD), the trigger-synchronization seemed sufficient to produce coherent point clouds that allowed the digital measurement of tree height and crown diameter(fig.12) (Petitpas, 2011).As for low vegetation, the vertical precision allow the general identification of vegetation groups.But with very high resolution images, the trigger-synchronization was not sufficient due to the move of the small branches and leaves in the wind.
In regards to theses promising results, a new test field campaign should be planned soon to validate the improvements of our new stereo rigs.

CONCLUSION
We have presented in this article the operational constraints due to the studies of natural environments, that come in addition to those implied by the scientific needs of the end users and the use of light UAV in LARS context as acquisition platforms.We have then presented two payloads designed to meet these constraints.
A particular attention has been granted to the synchronization of the acquisition, as it proves to be a technological lock for stereoscopic processing of close range images of natural scenes.The  Petitpas).Bottom: 3D mesh from stereo allow the identification of overground structures (here, low bushes of heather).
different solutions were explained with their limits and advantages.Finally, we have presented a qualitative analyse of the use of LARS images for a biodiversity monitoring study in a transition area between forest and moor.

Figure 2 :
Figure 2: Top: 3D reconstruction obtained using unsynchronized images (about 1 second delay), Bottom: 3D reconstruction obtained using synchronized images (to the hundredth of a second).

Figure 3 :
Figure 3: The developped payloads.Top: the tri-camera sensor, used on the Faucon Noir UAV.Bottom: the binocular stereoscopic rig, currently used on an underwater robot (sealed housings can be removed when the rig is used on a flying vehicle).

Figure 4 :
Figure 4: Different synchronization schemes.A full synchronization requires a trigger, a clock, a delay and an acquisition synchronizations.

Figure 5 :
Figure 5: Visual quality check of the synchronization of the uEye stereo rig on bushes in windy conditions.

Figure 8 :
Figure 8: : Automatic mosaicking of the three images acquired simultaneously by the tri-cameras system.

Figure 9 :
Figure 9: Very high resolution image with millimeter GSD that allows visual identification of species by botanists.

Figure 10 :
Figure 10: An example of fine species land-cover that can be made with the high resolution mosaics produced (courtesy of S. Mobaied).

Figure 12 :
Figure 12: Stereo reconstruction.Top: 3D cloud points from the stereo couple used for dendrometric measures (courtesy of B.Petitpas).Bottom: 3D mesh from stereo allow the identification of overground structures (here, low bushes of heather).

Table 1 :
Weight repartition of each systems (in grams).
Thus their are capable of full synchronization.The configuration is done by software: by disabling automatic settings and placing the acquisition loop of