UAS BASED TREE SPECIES IDENTIFICATION USING THE NOVEL FPI BASED HYPERSPECTRAL CAMERAS IN VISIBLE, NIR AND SWIR SPECTRAL RANGES

: Unmanned airborne systems (UAS) based remote sensing offers flexible tool for environmental monitoring. Novel lightweight Fabry-Perot interferometer (FPI) based, frame format, hyperspectral imaging in the spectral range from 400 to 1600 nm was used for identifying different species of trees in a forest area. To the best of the authors’ knowledge, this was the first research where stereoscopic, hyperspectral VIS, NIR, SWIR data is collected for tree species identification using UAS. The first results of the analysis based on fusion of two FPI-based hyperspectral imagers and RGB camera showed that the novel FPI hyperspectral technology provided accurate geometric, radiometric and spectral information in a forested scene and is operational for environmental remote sensing applications.


INTRODUCTION
The Unmanned airborne systems (UAS) are increasingly utilized in wide variety of environmental monitoring applications. Novel lightweight Fabry-Perot interferometer (FPI) based, frame format, hyperspectral imaging in the spectral range from 400 to 1600 nm offers a completely new possibilities for measuring and analysing the forest canopy. In this work, our objective is use the hyperspectral data for identifying different species of trees in a forest area.
The measuring system includes both UAS-and ground stations. The main payloads in the hexacopter type UAV are cameras, irradiance spectrometer and GPS receiver. The cameras were FPI based hyperspectral cameras and Samsung NX300 RGB camera. This investigation was performed using two novel spectral cameras, one of which operate in VIS (visible light) and NIR (near-infrared) areas (400 to 1000 nm) and is developed by VTT Technical Research Centre of Finland Ltd and Rikola Ltd. The second one collects data in SWIR (shortwave infrared) areas (1100-1600 nm) and it was developed by VTT. The VNIR (visible and near-infrared) camera is completely new prototype; in contrast to previous versions operating in the wavelength range of 500-900 nm Honkavaara et al., 2013;Näsi et al., 2015;Oliveira et al., 2016;Tommaselli et al., 2016), the new camera covers range of 400-1000 nm. The SWIR camera has been experimented in recent studies by Mannila et al. (2014) and Honkavaara et al. (2016), but is used for the first time in the forested environment. The FPI camera technology has been commercialized by Rikola Ltd. (www.rikola.fi).
When using the FPI technology, the hyperspectral data cube is formed in the time-sequential imaging principle. When using this technology on a moving platform, each band in the data cube exposed to a different air gap value has a slightly different position and orientation, which has to be taken into account in the post-processing phase.
In the remote sensing of forests, aerial imagery is often used for estimating stand level forest characteristics. The usual variables of interest are e.g. volume of growing stock and tree species composition. However, the accuracy of tree species recognition has typically not been very high when using aerial imagery. Hyperspectral sensors capable of discriminating larger number of spectral bands and wider spectral range provide one option for improving tree species identification, which is an important task in forest inventory.

UAS:
The UAV was a hexacopter with Tarot 960 foldable frame. Diameter of the rotors is 450 mm and KDE4014XF-380 electric motors were used. Autopilot is Pixhawk equipped with Arducopter 3.15 firmware. Payload of the system is 3-4 kg and flight time 15-30 min (depending on payload, battery and conditions).
The central components of system are hyperspectral cameras (see next section). The other main components are on-board GPS (RasPiGNSS, NV08C-CSM) for collecting UAV position trajectory, Ocean optics irradiance spectrometer USB 2000+ (OOU2000) and the cosine collector is stabilized to prevent tilting of the UAV from affecting the results of the irradiance measurement. Additionally Raspberry Pi2 on-board computer was used for collecting timing data for all devices, logging the GPS and for saving the OOU2000 spectra. Figure 1. The UAV and the instrumentation used in the UAV measurements The ground station is composed of reference panels for determining the reflectance level and equipment for irradiance measurements.

Cameras:
Two novel FPI-based spectral cameras were used, one of which operate in VIS (visible light) and NIR (nearinfrared) areas (VNIR; 400 to 1000 nm) and is developed by VTT Technical Research Centre of Finland Ltd and Rikola Ltd. The second one collects data in SWIR (shortwave infrared) areas (1100-1600 nm) and it was developed by VTT.
A Fabry-Pérot interferometer (FPI) consists of two mirrors that are facing each other. The distance between the mirrors -the "air gap" of the FPI -determines the wavelengths that interfere constructively and that are transmitted through the FPI, other wavelengths are reflected back. By varying the air gap the transmitted wavelength can be adjusted.
In the FPIs developed at VTT there are three small actuators (piezoelectric) that move one of the mirrors, and an electronic measurement of the distance between the mirrors at three different locations. A control loop constantly keeps the mirrors parallel to each other and at the desired separation. During hyperspectral capture the FPI subsystem progresses through a predefined sequence of air gap values, and at the same time the camera captures a frame for each air gap.
The new Rikola FPI-based camera for the full VNIR range, 400-1000 nm, is the first snapshot hyperspectral camera that is able to cover the whole VNIR range. The measured band can be selected with ±1 nm accuracy and the amount of bands is freely selectable. In addition, it is able to provide significantly sharper FWHM than other snapshot models, in some cases even below 5 nm FWHM is possible. It is equipped with custom optics having a focal length of 9.0 mm and an f-number of ~2.8. The camera has two RGB image sensor with an electronic shutter, one for VIS range, and one for NIR range. The image size can be selected between 1010 x 648 pixels and 1010 x 1010 pixels with a pixel size of 5.5 μm. The frame rate is typically 30 frames/s with 15 ms exposure time for 1010 x 648 image size. The maximum field of view (FOV) is ±18,5° in the flight direction and same ±18,5° in the cross-flight direction. The camera weighs about 720 g without battery. The camera can work independently taking images automatically to the compact flash card without any board computer, or it can be triggered by TTL-pulse from the autopilot according to the waypoints. The VNIR imager was used with settings for 36 bands from 409 nm to 973 nm with 10-15 nm FWHM in this study.
The SWIR range spectral imager consists of the commercial InGaAs camera -the Xenics Bobcat-1.7-320, the imaging optics, the FPI module, control electronics, a battery, a GPS sensor and an irradiance sensor (Mannila et. al, 2014). The Xenics Bobcat-1.7-320 is an uncooled Indium Gallium Arsenide (InGaAs) camera, with a spectral band of 0.9-1.7 µm and 320 x 256 pixels, and with a pixel size of 20 x 20 µm. The FPI, optics and electronics are designed and built at VTT. The focal length of the optics is 12.2 mm and the f-number is 3.2; the FOV is ±13° in the flight direction, ±15.5° in the crossflight direction, and ±20° at the format corner. The time between adjacent exposures is 10 ms plus exposure time; capturing single data cube with 32 bands and using 2 ms exposure time takes 0.384 s. The mass of the spectral imager unit is approximately 1200 g. The SWIR imager collected 32 bands from 1154 nm to 1576 nm with 20-30 nm FWHM depends on band in this study.
We used also RGB camera (Samsung NX 300) for capturing high spatial resolution stereoscopic images that can be used for stereoscopic 3D measurements to provide the 3D geometrical structure of the object.

Data processing
Hundreds of small-format UAV images were collected to cover the area of interest. Rigorous processing was required in order to derive quantitative information from the imagery. The processing of FPI camera images is similar to any frame format camera images; the major difference is the processing of the non-overlapping spectral bands. The processing chain has been developed by Honkavaara et. al. (2013). The FPI data processing line for the tree species identification contained the following steps: 1) Applying laboratory calibration corrections to the images. 2) Determination of the geometric imaging model, including interior and exterior orientations of the images. 3) Using dense image matching to create DSM. 4) Determination of a radiometric imaging model to transform the digital numbers (DNs) data to reflectance. 5) Calculating the hyperspectral image mosaics. 6) Extracting spectra of tree species 7) Classification In the following sections, the geometric (2, 3) and radiometric (1, 4, 5) processing steps and spectra extracting process (6) used in this investigation are described. The classification step was not yet applied.

Geometric processing:
Geometric processing determines the image orientations and creates point clouds and DSMs. Because the orientation of each band of the FPI data cube (typically 20-40 bands) would be computationally heavy, we have developed an approach that determines the orientations of selected reference bands and optimizes orientations of remaining bands to these reference bands. The reference bands are selected so that the temporal range of the images is covered as uniformly as possible. The geometric processing approach is described in details by Honkavaara et al. (2016) Agisoft Photoscan Professional (www.agisoft.com) software that is based on structure from motion algorithms was used to determine of the reference band orientations and create point clouds and DSMs. Firstly the Photoscan searches for common features in each image and then matches them. Then camera exterior orientations and camera parameters are refined using self-calibrating bundle adjustment. Sparse point cloud is then calculated. Then Photoscan calculates dense point cloud based on estimated camera locations and image information and DSM is calculated based on the dense point cloud.
VNIR and RGB images were processed simultaneously in one block using the Photoscan processing steps. The sparse point cloud was build and georeferenced using ground control points (GCP). Also camera parameters and camera locations were optimized based on ground control points. A dense point cloud was calculated using only RGB images to acquire higher density and better resolution. Same procedure was used to calculate scene for SWIR images For VNIR data set four bands and for the SWIR data set a total of five bands were used in the PhotoScan processing. For both data sets the sparse point clouds were calculated using the quality setting "high" with a key point limit of 40 000 per image and a number of tie points of 1000 per image. For the sparse point cloud an automatic outlier removal tool was used on the basis of the re-projection error (points that had >0.55 reprojection error level were deleted from VNIR scene and points >0.35 from SWIR scene). Also some displaced points were manually deleted (points below ground and on sky the dense point clouds were generated using "high" quality accuracy which uses original image size; mild filtering was chose to eliminate outliers. Finally, orientations of the bands that were not processed by PhotoScan were optimized to fit to reference bands.

Radiometric processing:
The radiometric modelling includes the sensor corrections, the atmospheric correction, correction for the illumination changes and other nonuniformities, and the normalization of illumination and viewing direction related nonuniformities (Honkavaara et al., 2013).
The sensor corrections for the FPI images include spectral smile correction, photon response nonuniformity correction (PRNU) and dark signal correction (Honkavaara et al. 2013. The PRNU and smile corrections were determined at the laboratory of VTT ). The dark signal correction is calculated using a black image collected right before the data capture. In this investigation, all these correction steps were used for the FPI VNIR camera. For the SWIR camera, only the dark signal correction was used; developing laboratory calibration procedures for the prototype sensor was not possible in this investigation.
The reflectance transformation from DNs to reflectance was carried out using the empirical line method (Smith and Milton, 1999) with the aid of the reflectance panels in the area. For the SWIR images, all reference panels with nominal reflectances of 0.03, 0.10 and 0.5 were used. For the VNIR images, the brightest panel was not used because it was saturated in most of the bands. Other radiometric corrections were not necessary because the imaging conditions were uniform during the data capture.
The orthophoto mosaics were calculated with a GSD of 20 cm using the FGI's radiometric block adjustment and mosaicking software (Honkavaara et al. 2013).

Campaign area and reference collection
Mustila Arboretum located in the municipality of Kouvola in South-Eastern Finland (60°44′N, 26°25′E) was used as the study area. The area has exceptionally large variation of various tree species, mainly originating from different parts of boreal and temperate zones of Northern hemisphere, nearly 100 conifer species and more than 200 broad-leaved tree species have been planted there. Mustila Arboretum was founded over 100 years ago, and thus, a large number of full-grown trees of various species occur in the area. Furthermore, trees of various species form homogeneous stands, which make the area suitable for testing tree species recognition both at stand level and individual tree level.
The total size of the arboretum is approx. 100 ha, and for this study an area of 50 ha was covered by hyperspectral imagery. The covered areas were selected on the basis of their tree species composition, specially targeting areas where trees compose homogeneous stands or tree groups as well as tree species of silvicultural importance. As a field reference, 47 test stands representing 25 different tree species were selected, delineated and examined in the field from the area covered by imagery. From the test stands 883 tree crowns were delineated on the basis of very high resolution aerial RGB imagery. The delineated trees include a number of intermingled (broadleaved) tree crowns that cannot be discriminated from the images.

Flight campaign
The flight campaign was carried out in September 2015 (Table  1). We chose four most interesting areas of Arboretum for flights ( Figure 2). The block structure was planned with same principle for every block such that the distance between flight lines was 30 m. The flying speed was 4 m/s and the flying height was 120 m, which resulted in a GSD of 0.03 m, 0.08 m and 0.20 m for the RGB, VNIR and SWIR cameras, respectively (Table 2). However, due to the terrain height variations and the canopy heights, the imaging distances were 70 m at smallest. During the flight campaign we also deployed 20 ground control points (GCPs) targeted with circular targets with a diameter of 30 cm. Their coordinates were measured using the virtual reference station real-time kinematic GPS (VRS-GPS) method.
The spectral settings of the FPI VIS/NIR and SWIR cameras were selected so that the spectral range was covered quite evenly (Table 3). A total of 32 spectral bands were collected by the SWIR camera in the spectral range 1100-1600 nm with the full width of half maximum (FWHM) ranging from 20 to 30 nm and with an exposure time of 2 ms. With the VNIR camera, 36 bands were collected on a spectral range of 400-1000 nm having a FWHM of 10-15 nm and an exposure time of 30 ms. Spectral and exposure settings for the cameras are shown in Table 3.

Geometric processing
The geometric processing was successful for all datasets. The re-projection errors were approximately 0.2-0.5 pixels ( Table  2). The DSM generation succeeded for all data sets (see example from Figure 3). The point density was approximately 320 points per m2 for every block. We also confirmed accuracy of geometric processing by comparing photogrammetric DSM to national airborne scanning (ALS) data by the National Land Survey of Finland (NLS). The difference between photogrammetric DSM and ALS data was less than 1 m on the flat areas for the most part of DSM.

Reflectance data
Reflectance mosaics were radiometrically uniform as expected because the illumination conditions were stable. However eight bands in the SWIR data (range 1360-1470 nm) were useless because of low level of signal due to the strong water absorption in this spectral range. Examples of mosaics for M1 area are shown in figure 4 (VNIR) and 5 (SWIR).  Examples of spectra of five trees are shown in in figure 5. They are averages of spectra of five individual trees.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLI-B1, 2016 XXIII ISPRS Congress, 12-19 July 2016, Prague, Czech Republic Figure 5. Spectra of five tree species

CONCLUSION
This investigation studied the performance of two novel lightweight Fabry-Pérot interferometer (FPI) based hyperspectral frame format camera prototypes in measuring the 3D geometry and spectral characteristics of forest canopy. The visible to nearinfrared range camera was the first frame format hyperspectral camera covering the spectral range of 400-1000 nm. The shortwave infrared camera prototype was used for the first time to map forest canopy. These two novel imagers offered a total of 60 bands in the spectral range of 400-1600 nm. Also a high spatial resolution consumer RGB camera was used to support geometric processing. More rigid image geometry as well as the possibility for stereoscopic measurements and multiple object views are important advantages of the frame format technology in comparison to conventional hyperspectral imaging technology based on pushbroom scanning. The results showed that the novel FPI hyperspectral technology provided accurate geometric, radiometric and spectral information in a forested scene and is operational for environmental remote sensing applications.