METHODS FROM INFORMATION EXTRACTION FROM LIDAR INTENSITY DATA AND MULTISPECTRAL LIDAR TECHNOLOGY

LiDAR is a consolidated technology for topographic mapping and 3D reconstruction, which is implemented in several platforms On the other hand, the exploitation of the geometric information has been coupled by the use of laser intensity, which may provide additional data for multiple purposes. This option has been emphasized by the availability of sensors working on different wavelength, thus able to provide additional information for classification of surfaces and objects. Several applications of monochromatic and multispectral LiDAR data have been already developed in different fields: geosciences, agriculture, forestry, building and cultural heritage. The use of intensity data to extract measures of point cloud quality has been also developed. The paper would like to give an overview on the state-of-the-art of these techniques, and to present the modern technologies for the acquisition of multispectral LiDAR data. In addition, the ISPRS WG III/5 on ‘Information Extraction from LiDAR Intensity Data’ has collected and made available a few open data sets to support scholars to do research on this field. This service is presented and data sets delivered so far as are described.


INTRODUCTION
Since its appearance in the latest '90s, LiDAR (Light Detection and Ranging) has consolidated as a major technique for the direct acquisition of 3D data (Vosselman & Maas, 2010).Notwithstanding the image-based photogrammetric techniques are quite competitive in many fields, thanks to Structure-from-Motion and dense surface matching algorithms (Barazzetti et al., 2009), the role played by laser scanning (LS) for topographic and 3-D modelling surveying is relevant at different scales and from multiple platforms: airborne (ALS), terrestrial (TLS), mobile (MLS), UAV LS, portable and hand-held LS.Satellite laser altimetry may complete the options, though based on a different data acquisition mode.
In the last decade, several studies have investigated the exploitation of the laser intensity returns as additional information to extend the use of LiDAR for classification purpose, often corroborated by the integration to other data types.Due to the typical high-resolution of laser scanning data, intensity is potentially a very important data source for classification problems, attracting the interest of the scientific community as described in the following sections.This potential interest has motivated the development of multispectral airborne lidar technology to expand vertical applications in high density topographic surveying, to enable applications such as land cover classification, shallow water bathymetry, among others.
After discussing the current state-of-the-art of sensor technology (Sect.2), in Section 3 the literature on the radiometric calibration and intensity normalization is reviewed.Indeed, while spatial aspects related to geo-referencing are shared with the standard data acquisition, the intensity data need to be specifically pre-processed before they are ready to be applied.These aspects are even more important when more LiDAR intensity data sets have to be merged, e.g., because of the presence of multiple strips or scans in the same project.Section 4 will report several applications organized per research domains.In Section 5 an innovative and breakthrough application of the intensity recorded by TLS to serve as quality measure is discussed.On 2017 the ISPRS Working Group III/5 on "Information extraction from LiDAR intensity data" has made available two data sets to enable scholars to work on such a type of data (Sect.6).Finally, Section 7 draws some conclusions and highlight new perspective for future and developments.

SENSOR TECHNOLOGY
LiDAR systems directly provide the 3D coordinates of the mapped surface at high density.The basic principle of operation relies on range measurements, i.e., the determination of the distance between the laser firing point and the footprint on the mapped object, which is derived based on the laser pulse travel time (Kersting, 2011;Vosselman & Maas, 2010).Range measurement can be operated in hardware, using for instance a constant fraction discriminator (CFD) and time interval meter, or by digitizing the received signal and applying any of a number of range detection algorithms to the output (Kashani et al., 2015a).Besides a laser ranging unit, lidar systems also entail a scanning unit allowing the data collection in a strip-wise fashion.The scanning mechanism usually consists of an oscillating mirror, which produces a zigzag pattern.Rotating polygon, palmer scan, and fibre scanner are examples of other scanning mechanisms (Wehr & Lohr, 1999).In the case of mobile sensor platforms, the integration with a GNSS/INS unit onboard which is used for georeferencing, i.e., for deriving the coordinates of the scanned object points relative to the mapping frame (Vosselman & Maas, 2010).
In addition to the geometric information (i.e., the 3D coordinates of the mapped points), LiDAR systems also can record the intensity (strength) of the backscattered laser signal.Most commercial topographic LiDAR systems are commonly monochromatic systems, i.e., operating with single laser wavelength in the near infrared (NIR -1064 nm).LiDAR systems for bathymetric purposes consist of dual-wavelength, which not only operate in the NIR (1064 nm) to measure the water surface but also at blue-green region (532 nm) for penetrating the water surface to a certain depth.Optech CZMIL Nova and Leica HawkEye III are examples of such bathymetric ALS systems.
In 2014 the first multispectral ALS system was released, i.e., Optech Titan (www.teledyneoptech.com).This sensor incorporates three independent laser beams at different wavelengths: 1550 nm in intermediate IR (Channel 1), 1064 nm in NIR (Channel 2), and 532 nm in visible green band (Channel 3). Figure 1 illustrates the spectral signature (reflectance properties) of different objects such as water, soil and vegetation for different regions of the electromagnetic spectrum, and the Titan operating wavelengths.

RADIOMETRIC CORRECTION AND NORMALIZATION
In order to be exploited, LiDAR intensity data need to represent the "real" scattering properties of the target surface.This condition is important either for target classification and for comparing/merging data obtain from different instruments and missions or acquired from multiple locations in the case of TLS/MLS.According to Höfle & Pfeifer (2007), the radiometric calibration and correction (RCC) of intensity data can be operated by two different approaches.In the empirical methods some statistics are used to minimize noise in intensity data.The physical methods are based on the use of the radar (range) equation, and on the knowledge of those factors that may have influenced the recorded intensity values (i.e., flying height, range, incidence angle, scan angle, terrain slope and aspect, sensor aperture size, surface moisture, automatic gain control on the backscattered intensity, reflection model, and atmospheric attenuation), see Wagner (2010).While more details on the methods for absolute calibration can be found in the literature (see Kaasalinen et al., 2011 andYan et al., 2015), all authors agree that the RCC leads to an improvement of land cover classification results in the average order of 10%, with peaks up to 31.2%.More details about the impact of the output product when using LiDAR intensity data after radiometric and geometric correction can be found in Habib et al. (2011), the latter also necessary to define the correct position of target locations (Yan et al., 2012).Other authors have followed the approach of radiometric normalization to compensate the radiometric differences of laser intensity in the case of overlapping strips (Jutzi & Gross, 2010;Yan & Shaker, 2014, 2016, 2017).These methods rely on the comparison of intensity values in common areas.

REVIEW OF APPLICATIONS WITH LIDAR INTENSITY
LiDAR intensity has been used in a large variety of applications related to geometric and radiometric aspects (or a combination of both).Applications range from basic activities where intensity is used to "colourize" aerial or terrestrial laser scans (point clouds) or images derived from scans.The first basic application is the simple visualization of the acquired data, especially from those sensors that do not have an associated RGB camera able to provide the "real" colour to point clouds (Barfoot et al., 2016).Although this task is quite trivial, the advantage of using the intensity-based colour information has a primary role in several projects, especially when RGB information is not available.The intensity-based colour approach was then extended to identify common features between multiple scans, resulting in new algorithms for matching and registration of point clouds.Such approaches include scan-to-scan or image-to-scan registration (Böhm & Becker, 2007;González-Aguilera et al., 2009;Kang et al., 2009;Alba et al., 2011;Altuntas, 2011;Barnea & Filin, 2011;Weinmann et al., 2011;Parmehr et al., 2016;Lohani & Sasidharan, 2017).Even though these methods have many limitations in practical applications, they represent potential useful alternative to traditional registration methods.Some other applications of LS intensity for evaluating the quality of geometric data will be discussed in Section 5. LiDAR intensity has been also used in computer vision problems related to object identification (Mioulet et al., 2017), segmentation (Umemura et al., 2017), localization (Wei et al., 2017), and deep learning, (Asvaldi et al. 2017), among the others.The remainders applications can be generally grouped into specific domains.

Land use classification
After Song et al. (2002), who proposed the use of LiDAR intensity for land cover classification, many authors have dealt with this task on the basis of ALS data.In particular, the addedvalue of using this type of observations is due to the small footprint, which gives the chance to map land cover at highresolution.This option is particularly important to study urban areas, for example to detect those regions covered by vegetation (Höfle et al., 2012).In Yan et al. (2015) a review of applications and processing methods related to this field is reported.While monochromatic LiDAR sensors have been used in the most case studies, recently multispectral sensors (see Sect. 2) have been applied, as reported in Morsy et al. (2017a).Also applications in non-urban areas have been proposed, such as the discrimination between water and land (Brzank et al., 2008;Morsy et al., 2018), the salt-marsh characterization (Collin et al., 2010), as well as several other specific studies which will be described in the following thematic subsections.Usually, intensity is not used alone but combined with other types of information, such as RGB colour (Cheng et al., 2017) height values (e.g., Morsdorf et al., 2010;Zhou, 2013;Zhou & Li, 2017), and LiDAR ranges (El-Ashmawy & Shaker, 2014).In Mesas-Carrascosa et al. (2012) and Yi et al. (2016), LiDAR intensity data are combined with multispectral images for classification purpose.

Different classification methods have been used along with
LiDAR intensity data.El-Ashmawy et al. ( 2011) discuss the use of pixel-based vs object-based classification techniques.In Zhou (2013), the integration of LiDAR height and intensity along with an object-based classification technique was demonstrated to be more efficient for urban land cover than using standard mediumresolution multispectral imagery.Results were comparable to the ones obtainable from multispectral high-resolution images and a DSM, see also Zhou & Li (2017) on the same topic.

Geology and geomorphology
Laser scanning has evolved into a widely used tool for research and applications in geosciences that require geometric and structural information about the Earth surface and processes shaping it.Raster-DTM and DSM have been in focus the last decade (Höfle & Rutzinger, 2011) 2017) on TLS in Earth sciences reveals that also the combination of TLS intensity and hyperspectral imagery (HSI) is gaining in importance for geologic feature mapping, which is also confirmed by new method developments for fusion of ALS and HSI (Brell et al. 2017).

Hydrolology
Apart from bathymetric LiDAR, topographic LiDAR intensity was widely investigated in hydrology to map water bodies and inundated areas such as flood-affected regions.In an early study, Höfle et al. (2009)

Glaciology and snow research
LiDAR intensity was investigated to map and retrieve different snow and ice facies and properties.Höfle et al. (2007) applied ALS intensity to map different glacier surface classes (snow, firn and ice) using point cloud-based segmentation and classification.Kaasalainen et al. (2008) found out that TLS intensity is affected by snow cover wetness.Joerg et al. (2015) calibrated ALS intensity data to derive a spatially distributed albedo proxy map of the glacier, which is needed as input for energy and mass balance models.In-situ measurements (albedometer, spectroradiometer) served as reference in this study to compute broadband albedo of the glacier surface from ALS intensity data.Podgorski et al. (2018) investigated calving events of a tidewater glacier in Antarctica with TLS intensity.They could classify different ages of glacier surfaces, i.e., different timing of ice face exposure.Median intensity correlated most with the ice exposure time with an explanation of 60% of the variability in intensity between calving events.

Vegetation studies in Agriculture, Ecology and Forestry
As the range of wavelength covered by LiDAR is 800-1500 nm, it is a powerful instrument for analysing vegetation structure.
LiDAR intensity has been investigated to derive biogeochemistry descriptions of vegetation (e.g., foliar nitrogen and chlorophyll photosynthesis) and ecologically relevant geographic parameters (e.g., drought stress, plant phenology, leaf area distribution, etc. -see Eitel et al. 2016).The main assumption is that plant parameters and the geometry of plant compartments alter the strength of backscatter due to changing reflectance and/or changing target area within the footprint.This fact can be used to empirically derive plant properties that are perfectly located in a topographic reference frame in contrast to passive spectral measurements.2010) adopted TLS for simultaneous measurement of plant structure and chlorophyll content.
Among the applications related to vegetation, a great attention is paid to agriculture.LiDAR is also one of the technologies usually used together with assisted GPS navigation, robotics, and drones (among others), for precision agriculture, i.e., the use of technology to optimize costly fertilizer and to ensure more efficiency and sustainability (Tamirat et al., 2018).Typical examples of practical applications are those related to the determination of soil and crop type Carranza & Blanco, 2015), agricultural planning and management, yield forecasting, crop analysis (Eitel et al., 2014), crop damage identification (Alberto et al., 2014), mapping of irrigation paths, and other applications related to field management.

Buildings and infrastructures
Applications based on LiDAR intensity for structures and infrastructures were proposed in the field of damage detection by González et al. (2010), Kashani (2014), Kashani & Graettinger (2015), Kashani et al. (2015b), andHou et al. (2017).Usually, the proposed approach is based on the use of well-known algorithms (or enhanced versions) for different preliminary tasks, such as classification or feature extraction.Then, specific application-based algorithms are used to detect damages, for example, after natural disasters (hurricane, earthquake, floods, tsunami, tornado), where major damages appear and can be found from the analysis of pre-and post-event information with an approach based on direct comparison of the two stages.In some cases, the analysis has to be conducted only on the post-event data because of the lack of data sets before the event.Identifying sudden changes in texture or other geometric and radiometric features allows to highlight possible damages.

LIDAR INTENSITY AS A QUALITY MEASURE
One of the most frequently asked questions in Geodesy, regardless of which sensor has been deployed, is related to the achievable precision of measurements that are also referred to as stochastic characteristics.This information is vital in order to weigh observations in the stochastic model of a parameter estimation, to reject outliers or to draw statistically sound conclusions for instance in deformation measurement.While the computation of stochastic measures for established geodetic instruments is well understood, this issue is still in the spotlight of current research when it comes to reflectorless rangefinders that form the key component of laser scanners.Hence, the following of this section has a thorough look at existing contributions on the subject of influencing factors onto the rangefinder precision of laser scanners as well as how these influences can be combined to an all-embracing stochastic model.Once the stochastic characteristics of an applied scanner are known, this information can be used to compute stochastic point clouds.This means that the individual precision is known for every single point within a dataset.
The identification of influencing factors onto the precision of reflectorless rangefinders is of great interest ever since the emergence of laser scanners.One of the first notable contributions on the subject was made by Böhler et al. (2003) who investigated the influence of different reflective characteristics.Furthermore the impact of increasing object distance was of interest.Vögtle et al. (2008) revealed notable differences among common building materials in terms of range noise for a TLS.In addition, noteworthy variations in terms of precision have been discovered between samples that were captured during the day or at night.Soudarissanane et al. (2011) approach the issue from a geometric point of view by analysing the effect of the angle of incidence.It was shown that the impact of interest increases for rising incidence angles.Interestingly, a stringent explanatory pattern can still be found in current research, see, e.g., Ozendi et al. (2017) and Bolkas & Martinez (2018), that assign various influences into certain categories.In summary, the most relevant influences can be tributed to the survey configuration as well as the local radiometric characteristics of the object.
Detailed knowledge about the impact of all these influencing factors onto laser scans is undeniably of vital importance.However, not a single contribution was able to mathematically tie these relevant influences together which would result in an allembracing stochastic model.Probably the biggest challenge in this context is bound to the consideration of radiometric characteristics.Yet numerous authors have reported significant effects onto the range noise for samples with different radiometric properties, what they have not mentioned is, how information about these properties can be captured or modelled.Spectrometers, for instance, are appropriate sensors to gather radiometric information at a certain wavelength, but they are not feasible to be applied in practice to analyse a point cloud collected by TLS.Hence, an alternative methodology has been proposed in Wujanz et al. (2017) that is based the analysis of raw laser intensity values.The general idea is based on the fact that laser scanners are photometric devices that convert an optical signal that is reflected by an object's surface into ranges.To this purpose, photo diodes are used that also deliver the intensity values as a by-product.Generally, the degree of range noise is bound to: (i) the strength of the incoming signal; and (ii) the individual characteristics of the applied photodiode.Wujanz et al. (2017) argue that all aforementioned influences cause a deterioration of the emitted signal that consequently yields to a loss of the precision -yet a causal separation of influencing factors cannot be However, if one has exact knowledge about (i) and (ii), then some conclusions about the achieved precision of individual measurements can be drawn.Since some manufacturers of laser scanners allow to export raw intensity values, such as Zoller + Fröhlich or Riegl just to name a few, the question arises how to derive (ii).The first option to achieve this is based on repeated distance measurements under varying acquisition configurations by solely using the rangefinder of a TLS.Stochastic measures are generated by computing the mean of raw intensity values as well as the standard deviation of ranges.
Figure 2 illustrates some results gathered in different experiments with a phase-shift Zoller + Fröhlich Imager 5006 h TLS, where raw intensity values are reported in horizontal direction while the standard deviation of ranges is associated to the vertical axis.Black circles stem from an experiment where planar panels with varying radiometric properties have been scanned under normal incidence angles and varying distances.The blue circles originate from measurements onto various building materials which have been captured under different incidence angles and ranges.The red circles have been also acquired under different incidence angles at two distances yet by usage of the aforementioned panels.It is obvious that a systematic run emerges -regardless of which influencing factor has been altered.Hence, these values can be used to estimate a function that finally forms the stochastic model of the rangefinder.Once the stochastic behaviour of a laser scanner is known, the deterministic information in form of the original point cloud can be enriched by stochastic information (see Wujanz et al., 2018).As a result, a stochastic point cloud emerges; an example is depicted in Figure 3.The upper right diagonal depicts an intensity-coded point cloud.Based on the radiometric information a stochastic information can be computed as illustrated on the lower left part of the figure.

CONCLUSIONS
From the current state-of-the-art of research on LiDAR intensity in manifold fields one can conclude that using this kind of observation has potential for improving classification over urban areas, mapping land cover (e.g., water areas) and bio-/geophysical parameter (e.g., water, chlorophyll and nitrogen content) retrieval, as clearly demonstrated in a multitude of studies.A major strength is the direct location of the "backscatter measurement" in topographic space -even in complex objects such as vegetation -compared to all passive spectral approaches.
Several studies showed that the combination of geometric and intensity information leads to improved results.Radiometric correction for distance and also angle effects is increasingly applied in recent studies.Still, several challenges need to be addressed and solved in the future: (1) transfer from controlled research in the lab with TLS to applications for large areas with ALS is needed because geosciences requires geographic data input; (2) correction of the intensity angle effects is still a chicken-and-egg question because a lot of information about the scattering characteristics of the surface/object of interest is required beforehand; 3) integration of LiDAR point clouds (3D coordinates+intensity) with hyperspectral imagery reveals a lot of opportunities but still demands for new methodologies for data fusion and information extraction; 4) hypertemporal time series of point clouds including intensity are still hardly exploited (Eitel et al. 2016) and no software tools for processing are available.New solutions are offered by multi-spectral LiDAR sensors, whose development and diffusion is expected to foster even more the exploitation of laser intensity data.Interesting applications may derive from the use of this kind of observation as a quality measure for simultaneously recorded geometric data.Last but not least, the availability of free data sets may help students and scholars to start doing research on this topic.

Figure 1 .
Figure 1.Spectral signature of different objects.Vertical bars represent the operating beam wavelengths of Optech Titan multispectral LiDAR (www.teledyneoptech.com).

Figure 3 .
Figure 3. MLS Data Set 1: the path followed by the sensor and the position of three subsets (on the left); a sample of the data set (on the right).

Figure 4 .
Figure 4. Multispectral ALS Data Set 2: outline of LiDAR strips (on the left); a sample from Data Set 2 (on the right).
Lang & McCarty (2009))6)iation) were jointly used with geometry in a region-growing procedure to delineate the river water surface.Malinowski et al. (2016)could even differentiate between different levels of water and grass coverage within single full-waveform ALS footprints using backscattering coefficient derived via radiometric calibration.The higher the water coverage, the lower the backscatter signal.Roelens et al. (2016)made use of airborne LiDAR intensity as proxy for NIR reflectance to support the extraction of cross sections (width, cross-sectional area) and water levels of vegetated ditches from the point cloud.The normalized difference water index (RGB and LiDAR intensity) was input for the classification of the presence of water and vegetation in ditch profiles.A similar objective of mapping channel networks from airborne LiDAR was presented byHooshyar et al. (2015).They mapped wet channels by integrating elevation and intensity of ground points via empirically derived thresholds on intensity and subsequent edge detection in the intensity image.For geophysical parameter retrieval, TLS intensity proved to be a suitable method to detect surface moisture and derive spatial patterns of the scanned surfaces in aeolian environments, seeNield et al. (2014).Lang & McCarty (2009)investigated wetland hydrology.
applied radiometrically corrected ALS data and modelled laser shot dropouts to segment and classify rivers in point clouds.Intensity-based variables of a local point neighbourhood (e