A REVIEW OF HYPERSPECTRAL IMAGING IN CLOSE RANGE APPLICATIONS

Hyperspectral imaging is an established method for material mapping, which has been conventionally applied from airborne and spaceborne platforms for a range of applications, including mineral and vegetation mapping, change detection and environmental studies. The main advantage of lightweight hyperspectral imagers lies in the flexibility to deploy them from various platforms (terrestrial imaging and from unmanned aerial vehicles; UAVs), as well as the high spectral resolution to cover an expanding wavelength range. In addition, spatial resolution allows object sampling distances from micrometres to tens of centimetres – complementary to conventional nadir-looking systems. When this new type of imaging device was initially released, few instruments were available and the applicability and potential of the method was restricted. Today, a wider range of instruments, with a range of specifications, is available, with significant improvements over the first generation of technology. In this contribution, the state-ofthe-art of hyperspectral imaging will be reviewed from a close range measurement perspective, highlighting how the method supplements geometric modelling techniques. An overview of the processing workflow, adjusted to the more complex close range imaging scenario will be given. This includes the integration with 3D laser scanning and photogrammetric models to provide a geometric framework and real world coordinate system for the hyperspectral imagery. * Corresponding author


INTRODUCTION
Hyperspectral imaging (HSI) is an established method for surface material and mineral mapping, which has been applied for decades form airborne and spaceborne platforms.With the release of compact hyperspectral image sensors during the last ten years, HSI has become highly relevant for close range applications.The fact that HSI can now be applied from different platforms and in various scanning scenarios and setups, such as tripod-based (Figure 1, e.g.Kurz et al. 2013), in laboratory setups (e.g.Zaini et al. 2014), on UAVs (e.g.Zarco-Tejada et al. 2012) or ultralight aircraft (e.g.Lausch et al. 2012), has resulted in an increase in interest in this method.A number of new and innovative applications have been demonstrated indicating the high potential of the methods in disciplines such as geosciences (e.g.Kurz et al. 2012b, Kurz et al. 2013, Murphy et al. 2015, Zaini et al. 2014), environmental applications (e.g.Jay et al. 2016), medical applications (e.g.Lu and Fei 2014), biosciences (Gowen et al. 2015) and many others (e.g.Edelman et al. 2012, Liu et al. 2015, Zarco-Tejada et al. 2012).As an emerging area, with ongoing research into acquisition best practice, processing and classification algorithms, the aim of this paper is to assess the state-of-the-art of HSI in close range applications.The article reviews undertaken research on this topic including a brief overview of processing workflows and scanning scenarios, based on existing studies in the literature.The outcome is a discussion on advantages and current limitations of the method, as well as future work required to improve reliability, robustness, efficiency and practicability of HSI in close range applications.

INSTRUMENTATION AND SCANNING SCENARIOS
Sensor and technology development is rapid, and it is outside the scope of this paper to comprehensively review the available instrumentation and sensor specifications.However, current hardware can be categorised according to basic specifications, such as spectral range, spectral and spatial resolution and data acquisition technique, which are often essential for applicability to specific application domains.
The measured spectral range defines which types of material can be efficiently analysed, as this depends on where in the electromagnetic spectrum diagnostic spectral properties occur.Therefore it is important to choose an appropriate spectral range according to the application and materials of interest.Early close range HSI instruments covered mainly the visible (VIS) and near infra-red (NIR) spectral ranges (e.g.Murphy et al. 2008), current technology has developed to cover the shortwave infra-red (SWIR; e.g.Kurz et al. 2013, Murphy et al. 2015) and thermal infra-red (TIR; e.g.Schlerf et al. 2012) ranges.The spectral resolution of these instruments provides spectral sampling usually between 3-7 nm for the VIS, NIR, and SWIR spectral ranges, and 0.25-250 cm-1 for the TIR.These spectral resolutions are high considering that diagnostic absorption properties are usually wider than 20 nm; thus spectral resolution is seen as appropriate for detailed analysis of diagnostic spectral properties.
The scanning scenario and instrument setup is another important element in close-range HSI, which differs from nadir imaging, and is flexible and adjustable to application requirements.The majority of close range hyperspectral imagers are pushbroom scanners, requiring movement either of the sensor line or of the object to acquire images.A recent innovation, at least for the VNIR spectral range at the present time, is frame hyperspectral cameras, sometimes also termed as snapshot hyperspectral camera (e.g.Bareth et al. 2015).Frame cameras provide a number of key benefits, as they can be used analogously to conventional digital cameras.Thus imaging geometry becomes the conventional pinhole model rather than specialised geometrical models (Kurz et al. 2011), and it is possible to image moving objects, as an entire image frame is captured at once.Tripod-based HSI allows imaging of vertical objects and cliffs (e.g.Kurz et al. 2013, Murphy et al. 2015).Like conventional digital imaging, hyperspectral imaging relies on recording reflected light as photons.This can be problematic for the SWIR range, as it relies on a strong infra-red light source.In some applications solar illumination is not possible, due to suboptimal face orientations or enclosed spaces.Artificial light sources have been successfully applied to scan, for example, subsurface tunnel faces (Kurz et al. 2012a).Miniaturisation has allowed increasingly-lightweight sensors to be produced.Lausch et al. (2012) applied HSI from an ultralight aircraft for multiscale vegetation studies.Ultralight hyperspectral sensors open up the possibility for UAV-based applications (e.g.Zarco-Tejada et al. 2012, Bareth et al. 2015).However, at the present time, such sensors are restricted to the VIS and NIR spectral ranges, as for frame hyperspectral sensors.
Hyperspectral imaging in laboratory setups extends analytical spectroscopy methods, which conventionally apply single pixel spectrometers.Such imagers allow high resolution hyperspectral data to be collected even for small specimens or objects, and at microscopic scales (e.g.Medina, et al. 2011).The advantage of laboratory setups are the controlled illumination conditions and relatively low atmospheric effects compared to other close range HSI scanning setups.Spatial resolution of millimetre (e.g.Speta et al. 2013, Zaini et al. 2014) to nanometre (e.g.Medina, et al. 2011) scale can be achieved.Due to the fact that HSI is non-destructive and allows time-efficient material analysis with high accuracy and reliability, laboratory HSI is becoming rapidly adapted in a broad range of applications (e.g.Edelman et al. 2012, Feng, et al, 2012, Gowen et al 2015, Lausch et al, 2012, Liu et al. 2015, Lu et al. 2014, Medina et al. 2011, Speta et al. 2013, Zeini et al. 2014).
Hyperspectral imaging technology has seen a large improvement and maturation over the last decade resulting in qualitative improved hyperspectral products and various sensor types and sensor specifications.This allows users to choose sensors and scanning scenarios appropriate to specific applications.

PROCESSING WORKFLOWS
Processing of close range hyperspectral imagery generally follows similar processing workflow as established for airborne and spaceborne data (Kurz et al. 2013).The processing workflow can be divided into three stages: i) Pre-processing to transfer at-sensor radiance into reflectance, correct atmospheric effects, compensate for illumination heterogeneity, and remove geometric and other image artefacts.ii) Spectral end-member extraction is frequently applied to retrieve spectrally-pure material spectra to interpret and understand material abundances in the image.These are used as reference spectra for subsequent classifications and spectral unmixing approaches.iii) Spectral mapping and classification to create material distribution maps.
Although this established processing workflow allows results to be obtained from close range datasets, there is still research to be conducted to refine and adjust workflows to close range imaging geometry, the specific sensor types and scanning scenario.Instead of extracting material information directly form at-sensor radiance measurements, materials are often identified by their reflectance spectra which can be related to material intrinsic properties.To transfer at-sensor radiance measurements into surface-spectral reflectance, empirical methods such as Empirical Line Correction (e.g.Smith and Milton, 1999), which applies calibration panels with known reflectance, are widely used (e.g.Kurz et al. 2013, Murphy et al. 2015).Close range application allows in most cases easy access to the object to image, facilitating the logistics of placing calibration panels into image scenes.This is in contrast to airborne/spaceborne image acquisition where target placement must be carefully planned to coincide with flights and/or overpasses.Whereas empirical methods provide a time efficient and easy to use calibration method in close range imaging, they are also associated with number of shortcomings.It is assumed that calibration derived from panels is appropriate and representative for the entire scene.This assumption is not always true, as effects such as scanning geometry, illumination, or atmospheric irradiance caused by different scanning ranges can vary within a single scene.Image acquisition with push broom techniques can last few seconds to minutes depending on scanning parameters such as integration time or number of lines to acquire which also can result in temporary variation within a scene.Little research has been conducted applying physical modelling to extract reflectance images in hyperspectral close range scenes.Jay et al (2016) presents a physical approach to describe leaf reflectance observations, including the VIS, NIR and SWIR spectral ranges.To achieve optimal and efficient calibration and correction results in close range hyperspectral imaging, more work is required to adapt approaches used widely in airborne and spaceborne imaging.In addition, methods should potentially combine empirical and physical modelling-based methods.
In close range HSI using pushbroom sensors, a number of image artefacts such as bad pixels, striping, illumination inhomogeneity or adjacency effects caused by backscattering of nearby objects are frequently observed (e.g.Kurz et al. 2013).
To which level such effects require correction depends on the artefact intensity, later spectral mapping methods and the specific application.However, since image artefacts and noise can significantly hamper classification and spectral mapping results, image correction can be an important and challenging task.To remove noise and artefacts, methods known from image processing, such as Fourier filtering for de-striping, can be applied.Other methods have been developed that consider the high dimensionality of hyperspectral data.One example, the Maximum Noise Fraction transformation (MNF; Green et al. 1988), is frequently used to separate random noise in hyperspectral images, and is based on principal component analysis adapted to hyperspectral images.Since MNF transformation results in reduction of the dimensionality of the hyperspectral dataset, and therefore smaller datasets for efficient subsequent processing, it has been become an attractive processing technique in hyperspectral imaging.Several authors have also demonstrated how geometric data (see section 4) can be utilised for shadow correction and to compensate for illumination inconsistency and neighbourhood effects (e.g.Ramakrishnan at al. 2015) in close range HSI.
Reference spectra, frequently used as so called end-member spectra for classification input, can be extracted in different ways.Using statistical methods, such as those described in Aggarwal and Garg (2015), allow end-members to be extracted directly from the hyperspectral images.Although these methods provide spectrally pure end-members, they are often sensitive to image artefacts and interpretation, and may not be representative of real end-member material classes.Other possibilities take into account that areas and objects are often accessible in close-range applications.Reference areas for endmember collection can be marked prior to hyperspectral data collection and located within the hyperspectral imagery during data processing.Alternatively, geometric data such as photorealistic lidar/photogrammetric models can be used to locate reference areas after data fusion with the hyperspectral imagery (Kurz et al. 2012b, Kurz et al. 2013).In-situ spectral measurements using high resolution single pixel spectrometers are another possibility to create a reference spectral library of the materials of interest (e.g.Denk et al. 2015).While in-situ measurements provide high resolution and accurate reference spectra, it needs to be ensured that in-situ and image spectra are comparable due to data being collected with different instruments, spectral ranges and potentially under varying conditions.
For spectral mapping and classification, a wide range of methods and algorithms are available, well known from airborne/spaceborne HSI.In contrast to conventional RGB images, hyperspectral data provide a higher spectral dimension comprising detailed material information for image classifications.It is outside the scope of this paper to review all methods available.However, Spectral Angle Mapper (e.g.Kruse et al. 1993) classification has been become an attractive method due to the simplicity and low computation costs.Another important category of spectral mapping methods is spectral unmixing, which allows sub-pixel abundancies to be estimated (Keshava and Mustard, 2002).Spectral mapping and classification of hyperspectral images is an ongoing research topic to improve robustness and efficiency, and is not restricted to close range applications (e.g.Miao et al. 2014, Sabale andJadhav 2015).

INTEGRATION WITH TOPOGRAPHIC DATA
Hyperspectral imagery provides material information but, as with any digital image, without relating the content to a real world coordinate system.Linking hyperspectral imagery to a spatial reference system provides a number of advantages for pre-processing the hyperspectral data, as well as for analysing and presenting results (Buckley et al. 2013, Kurz et al. 2013).To combine material information with spatial geometric information, such as from terrestrial laser scanning or photogrammetry, several data integration approaches have been suggested.Kurz et al. (2011) integrates hyperspectral imagery with terrestrial laser scanning models.Many hyperspectral imagers operate as pushbroom line sensors mounted on a rotation stage.These cameras have to be modelled as panorama cameras with cylindrical image geometry for photogrammetric processing (Schneider and Maas 2006), in contrast to conventional frame cameras.Hyperspectral imagery is registered to the lidar coordinate system measuring ground control points within an image either manually or in an automatic manner (Monteiro et al. 2013, Sima et al. 2014).Once a hyperspectral image has been registered, classification maps can be spatially exploited and spectral information can be visualised in different ways, such as by colouring point clouds or through multiple layer texture mapping of 3D meshes (Figure 2, Buckley et al. 2013).Markelin et al. (2014) demonstrates 3D points cloud extraction from a UAV based hyperspectral frame camera applying selfcalibrating bundle block adjustment and global image matching algorithms.A point density corresponding to the pixel size of the hyperspectral imagery was possible.
Another solution to combine geometry and spectral information is achieved with innovative sensor technology.Hakala et al. (2012) demonstrated a full waveform hyperspectral lidar instrument for terrestrial laser scanning.With this instrument, spectral backscattering information can be extracted from an active laser pulse, resulting in topographic and spectral intensity information from a single measurement pulse.As technology improves over time this development looks promising and has the potential to solve illumination problems currently faced by passive HSI in close range applications.However at present time the spectral resolution with 16 spectral channels, 34 nm spectral sampling intervals and 19 nm FWHM is significantly lower compared to conventional passive hyperspectral imagers, and is limited to short distance.
Until now, most work has been focused on georectifying close range hyperspectral imagery and visualising hyperspectral products in a 3D real world coordinate framework.Geometric information can be also utilised for hyperspectral image correction during pre-processing as demonstrated by Ramakrishnan at al. (2015), where illumination invariances were compensated by illumination modelling based on geometric information retrieved from 3D terrestrial lidar data.More work is required to apply geometric information for image correction during with the aim of improving robustness and reliability of hyperspectral classification and analysis.
Figure 2. Terrestrial lidar model multi-textured with RGB photographs and hyperspectral material distribution image.

DISCUSSION AND CONCLUSIONS
Close range hyperspectral imaging is a technology-driven field of research.As noted by Kurz et al. (2011), hyperspectral technology has significantly improved since the release of the first compact hyperspectral imagers.Sensor quality, as well as spatial and spectral resolution, has increased.Most noticeable is the increasing range of sensors with different properties and specifications available on the market.This allows imagers to be chosen and scanning setups to be optimised to ensure high data quality for specific applications.However, further development is required before all spectral ranges can be deployed in different scanner scenarios, such as on UAVs.
One important aspect and strength of close range HSI is the capacity to provide multi-scale spatial material information from nanometres to decimetre resolution, which extends the spatial resolution conventionally achieved by airborne or spaceborne data (e.g.Lausch et al 2012).Hyperspectral imaging in laboratory microscopy setups has been become a routine method in disciplines such as biology, medicine, food safety or in geoscience laboratories, largely due to the non-destructive, non-contact manner and less requirements on sample preparation compared to alternative analysis methods such as XRD.Laboratory applications benefits from the controlled scanning and illumination conditions, which facilitate data collection, processing and analysis, compared to tripod or UAV-based imaging.However, more cross-discipline knowledge transfer between the different application disciplines is desired to provide solutions for common problems in data processing, analysis and visualisation.
At the present time, few software packages are available for hyperspectral image processing.Specific processing tools for hyperspectral close range imagery considering image geometry and sensor specifications are not yet implemented into off-theshelf software packages.This hampers the uptake of close range HSI for non-remote sensing experts and working groups with limited capacity for developing dedicated methods.Further work is required by the scientific community to extend appropriate processing tools for close range HSI.Enhanced processing is required to compensate atmospheric, illumination and geometric effects based on physical modelling, data fusion and sophisticated material mapping techniques.Data processing, visualisation and automation will benefit greatly from more cross-disciplinary research to integrate aspects and methods from different field of research, such as remote sensing, geomatics, computer vision and robotics.
During the last decade the general workflow for data acquisition, processing and analysis of ground based HSI has been established.A wide range of novel applications has been demonstrated and ground based HSI has been proven as a valuable complementary tool for spatial material mapping.It can be expected that with further developments, improved hardware, processing and analysis, ground based HSI will continue to mature as a valuable material mapping tool in a multitude of applications.

Figure 1 .
Figure 1.Tripod-based hyperspectral imaging for geological mapping of vertical cliff faces, showing HySpex SWIR-320m imager.The vertical scanning line of the sensor is moved on a rotation stage for image acquisition.