TEMPERATURE – EMISSIVITY SEPARATION ASSESSMENT IN A SUB-URBAN SCENARIO

In this paper, a methodology that aims at evaluating the effectiveness of different TES strategies is presented. The methodology takes into account the specific material of interest in the monitored scenario, sensor characteristics, and errors in the atmospheric compensation step. The methodology is proposed in order to predict and analyse algorithms performances during the planning of a remote sensing mission, aimed to discover specific materials of interest in the monitored scenario. As case study, the proposed methodology is applied to a real airborne data set of a suburban scenario. In order to perform the TES problem, three state-of-the-art algorithms, and a recently proposed one, are investigated: Temperature-Emissivity Separation ’98 (TES-98) algorithm, Stepwise Refining TES (SRTES) algorithm, Linear piecewise TES (LTES) algorithm, and Optimized Smoothing TES (OSTES) algorithm. At the end, the accuracy obtained with real data, and the ones predicted by means of the proposed methodology are compared and discussed.


INTRODUCTION
Hyperspectral Remote Sensing in the Long Wave InfraRed (LWIR), also known as Thermal InfraRed (TIR), provides useful information for geophysical and surveillance purposes.Object with a temperature higher than absolute zero, emits thermal radiations into the 8 -12 µm region of the electromagnetic spectrum.Radiance measurements made in this spectral window allow surface emissivity and temperature estimation, which are thermal-optical intrinsic properties of the surface and its physical state.Furthermore, chemical identification of the objects can be performed without an external light source, allowing day/night operations (Eismann, M. T., 2012).Material's emissivity and temperature are jointly combined inside the radiance measurements.The joint estimation of both physical quantities represents a challenging task: assuming that the atmospheric effects have been completely compensated, the estimation is an ill-posed problem because, for the given observed radiance at N channels, there will always be N+1 unknows (N emissivity samples and the surface temperature).Besides, the atmospheric downwelling radiance and surfaceemitted radiance are also coupled together through emissivities, making the estimation more complicated.This problem is known in the literature as Temperature -Emissivity Separation (TES) problem.
During the last decades, many scientists have approached this underdetermined problem from different perspectives.Empirical approaches exploit further equations or constraints, based on some a-priori information, such as smoothness characteristics of the spectra, combined with the presence of atmospheric features inside the signal spectral shape (Kealy, P. S. et al., 1990), (Watson, K., 1992).Instead, statistical approaches assume that the unknown emissivities and temperature can be modelled as a random vector with a given multivariate distribution (Ash, J. N. et al., 2016).Despite the many available approaches, it is not easy for the scientists or the final users to select the most effective algorithm (SRTES) algorithm (Cheng, J. et al., 2010), Linear piecewise TES (LTES) algorithm (Wang, N. et al., 2011), and Optimized Smoothing TES (OSTES) algorithm (Pivovarník, M. et al., 2017).These algorithms have already shown their effectiveness on the TES framework, but they have never been compared in a critical manner, i.e. exploiting a common data set and merit figures.The paper is organized as follows: in Section 2, the proposed methodology is introduced and deeply explained.In Section 3, a case study of a suburban scenario is shown.In Section 4, the experimental results are presented and discussed.

METHODOLOGY
In the planning of a remote sensing mission, it is crucial to choose the most appropriate TES algorithm among the ones available in literature.The choice is not trivial because the different strategies have been developed starting from different assumptions, and it is not straightforward to select the most appropriate algorithm for the given sensor/scenario combination.It is worth pointing out that TES algorithms work in an iterative manner, and are influenced in a complex way by many variables, such as the atmosphere composition, its temperature profile, the sensor spectral resolution and radiometric sensitivity, the spectral signature and temperature of the materials, etc. (Li, Z. L. et al., 1999), (Pieper, M. et al., 2017), (Ahlberg, J. 2017).Thus, TES algorithms performance cannot be computed analytically and a complete simulation of all the steps that, starting from the material emissivity, lead to the measured spectral radiance, and then to the ground leaving radiance, is required.In this framework, we present a methodology aimed to evaluate and predict the performance of TES strategies during the planning of a remote sensing mission, oriented to discover specific materials of interest in the monitored scenario.The proposed methodology is summarised in Figure 1: In this first step, we should recover the emissivity spectra representative of such materials.For this purpose, we have set up a data base gathering spectral emissivities of many materials.The spectra have been collected from public spectral libraries, such as ASTER Library (Baldridge et al., 2009), MODIS Library (Wan et al., 2005), and Urban Micromet -University of Reading Library (Kotthaus et al., 2014).
Step 2 -At-sensor radiance simulation The second step concerns the simulation of the at-sensor radiance, taking into account the expected target temperature and atmospheric profile, at the time of the planned mission.The atmospheric profile includes several parameters, such as pressure, temperature, aerosols, and gases contents, that can be changed to match the expected conditions of the scenario.Simultaneously to the atmospheric profile, also the acquisition geometry must be defined, by setting such parameters as altitude of the sensor and viewing angle.The at-sensor radiance   (  , ) can be expressed by the Radiative Transfer Equation (RTE) as follows (Li, Z., 1999) The atmospheric transmittance, upwelling and downwelling radiances, required in (1) and ( 2), are then obtained with the aid of a radiative transfer code.In this work, we used the current version of the MODerate resolution atmospheric TRANsmission (MODTRAN) computer code (Berk, A. et al., 2006).The at sensor radiance in (1), ( 2) is computed with a spectral sampling interval of 1 cm -1 , which is the smallest spectral sampling allowed by the MODTRAN code.
Step 3 -Sensor model The next step concerns the description of the sensor that will be employed in the planned mission.In a real environment, the sensor affects the measurements from various perspectives.A key factor is the sensor spectral resolution.In fact, the radiance   (  ) at the output of the k-th spectral channel is obtained by weighting the pupil plane radiance with the spectral responsivity function   (  ) of the channel.Such filtering operation is simulated as follows (Ahlberg, J. SRF smoothing may hide or reduce spectral features in the radiance spectrum thus strongly influencing the proper functioning of certain TES algorithms.
Another crucial factor, that may reduce the effectiveness of the temperature/emissivity retrieval, is the sensor radiometric sensitivity.Radiometric sensitivity is determined by random noise sources, which are usually quantified by the NEDT figure of merit, and by the "coherent" or non-uniformity noise induced by calibration errors in the gain and offset of each detector. Step

-Atmospheric correction
The fourth step of the proposed methodology is to simulate the atmospheric correction of the measured radiances.This step is a key point of the entire process, since it is possible to simulate the impact of errors in the atmospheric correction on the TES performance.In the literature, this issue is usually neglected and TES algorithms are evaluated starting directly from the groundleaving radiance (2), i.e. overlooking the path radiance and transmittance compensation.Errors in the atmospheric correction not only corrupt   ↓ (  ), but also (  ) and   ↑ (  ), introducing uncertainties whose consequences on the TES algorithms performance cannot be easily predicted.
Step 5 -TES algorithms This step regards the application of different TES algorithms, in order to evaluate the performance of various strategies and select the most suitable for the specific scenario under test.The TES strategies we consider are three state-of-the-art algorithms, and a recently proposed one: Temperature-Emissivity Separation '98 (TES-98) algorithm (Gillespie, A. et al., 1998), Stepwise Refining TES (SRTES) algorithm (Cheng, J. et al., 2010), Linear piecewise TES (LTES) algorithm (Wang, N. et al., 2011), and Optimized Smoothing TES (OSTES) algorithm (Pivovarník, M. et al., 2017).
The inputs of the algorithm are the ground-leaving radiance   (  , ),   ↓ (  ), and a first supposed maximum value of the emissivity   (grey body approximation).
The NEM module estimates the material temperature as the maximum brightness temperature.By inverting the Planck's law, with this temperature, the spectral emissivity is retrieved.The estimated emissivity is the input of the RM module, that provides the so-called beta-spectrum: where ′ ̅ (  ) is the mean value of the estimated emissivity.
The beta-spectrum preserves the shape but not the intensity of the emissivity.The rescaling process is provided by the MMD module, where an empirical relation between the minimum value of the emissivity   , and the difference between the maximum and minimum value of (  ) is used: The coefficients , ,  must be preliminary estimated in a calibration phase, taking into account the specific scenario/sensor (Gillespie, A. et al., 1998).

Stepwise Refining TES (SRTES)
The inputs of the algorithm are   (  , ),   ↓ (  ), and a certain number of spectral windows, chosen as the one more strongly influenced by the atmosphere.A pseudo-code of the algorithm is shown in Figure 2: The basic idea of the algorithm is to search for the right value of the emissivity in each spectral window.For further details, see (Cheng, J. et al., 2010).

Linear piecewise TES (LTES)
The inputs of the algorithm are   (  , ),   ↓ (  ), and M, i.e. the number of sections into which the wavelength axis is splitted.
A pseudo-code is shown in Figure 3: The basic idea of the algorithm is to reduce the number of unknowns making a linear piecewise approximation of   (  , ).For further details, see (Wang, N. et al., 2011).
OSTES represents an improved version of TES-98, in fact, they share both the RM and MMD modules.The main differences are in the first step: instead of the NEM module, the algorithm exploits a linear relationship between the brightness temperature   (  ) extracted from   (  , ), and the unknown emissivity.Figure 4 shows an example of this relationship.For further details, see (Pivovarník, M. et al., 2017).
Step 5 -Performance evaluation Performance evaluation take taking into account several indexes, that emphasize various aspects of the TES problem.Usually, the main indexes used are the mean error of the retrieved temperature, and the root mean square error (rmse) of the retrieved emissivity, defined as: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3/W3, 2017 Frontiers in Spectral imaging and 3D Technologies for Geospatial Solutions, 25-27 October 2017, Jyväskylä, Finland where   = center wavelengths; (  ) = real emissivity (from database); (  ) = reconstructed emissivity.
We want to emphasize that the methodology allows the operator to take into account all the possible error sources.A large amount of simulations can be performed following the proposed methodology, covering all the critical aspects of a remote sensing mission.

CASE STUDY
In this work, we illustrate the proposed methodology on a realistic case study.We selected as target materials, the ones shown in Figure 5, and took samples representative of their spectral emissivities from our data base (step1, in Section 2).The emissivities for the selected materials are shown in Figure 6, 7, 8, 9, and 10.

EXPERIMENTAL RESULTS
In this chapter, we show an example of application of the proposed methodology to the selected case study.Exploiting the emissivities chosen in step 1, and the expected material's temperatures for the scenario, we reconstruct the at-sensor radiance (1).The sensor responsivity is simulated using the nominal values for the TASI sensor.We assumed 32 spectral channels from 8 to 11.5 µm, with a Gaussian SRF having FWHM of 109.5 nm.In order to illustrate the dependence of the algorithms performance on different SNR values, and on atmospheric uncertainties, we set up Monte Carlo simulations.For each group of material samples, we generate 150 noise realizations at a fixed SNR, as defined in (Acito, N. et al., 2011) for the signal independent noise part, with values ranging from 40 to 50 dB (Ahlberg, J. 2017).For the fixed SNR, and for each noise realization, we evaluate the root mean square error (rmse), defined in (6), and the mean error on the emissivity and temperature retrieval, respectively.Finally, the two indexes are averaged over the 150 realizations.In the same manner, in order to evaluate the algorithms robustness to atmospheric uncertainties, we simulate an under/over-estimation of the nominal water vapour column, using scaling factors of 0.3, 0.6, 1.4, and 1.7.Furthermore, we add noise with a SNR of 45 dB (close value to the sensor nominal radiometric sensitivity), and generate 150 noise realizations as previously.The results are then compared with the ones obtained from the real data set.

Ideal case
Tables 2 and Table 3 show the rmse and mean error on the emissivity and temperature retrieval, respectively, in the case of no error sources, i.e., no noise or atmospheric uncertainties are added during the application of the methodology.This means that in (1) and (2), (  ) and T are the only unknowns: rmse  3. mean error on the temperature retrieval.In red the lower mean error for the given material.
The tables show that it is not possible to define the best algorithm for the specific scenario, without any other evaluation test, since different materials have different minimum values of the rmse and mean error.Furthermore, for the materials of interest, not always the minimum value of the rmse, given by the algorithm, corresponds to a minimum value of the temperature mean error, for the same algorithm.As shown in (Pieper, M. et al., 2017), different values of temperature bring to different performances of the algorithms, since the Planck's function moves its emission peak according to the Wien's displacement law.Distinct positions of the peak change the initial conditions of the algorithms.The behaviour of the algorithms can be explained by the fact that an over/under estimation of the emissivity (e.g.vertical rigid shift) can be compensated by an under/over estimation of the temperature, to the point that the two effects compensate each other.
We point out that, as expected, LTES over the steel samples is much more effective than TES-98 and OSTES.In fact, these two latter algorithms have been derived assuming that the spectral signature under test has an emissivity close to 1.As shown in Figure 7, the steel has a lower emissivity than the other materials, and the bond of the algorithms is not respected.

Impact of SNR on TES performances
Figures 11 and 12 show the rmse and the mean error on the emissivity and temperature retrieval, respectively, for different SNRs.Since the performances are quite similar for the tar paper, asphalt, concrete-cobblestone, and graveler, we present the mean rmse values and temperature errors, averaged among the four materials.The figures show that TES-98 and OSTES outperform the other two algorithms, and are quite insensitive to the noise level.Concerning the temperature retrieval, the bias is under 1K, which is a basic requirement for many geophysical and surveillance applications.SRTES is the most sensitive to the noise level.In fact, it is based on the presence of strong emission/absorption features into the signal spectral shape, thus, a corruption due to noise may conceal these features, and reduce the algorithm performance.Though TES-98 and OSTES perform similarly, OSTES seems to be better for high SNR.
Figures 13 and 14 show the rmse and the mean error on the emissivity and temperature retrieval, respectively, for the steel samples.As described in paragraph 4.1, if the spectral signature has low emissivity, TES-98 and OSTES are expected to make higher errors than LTES and SRTES.LTES seems to be the proper choice in case of low emissivity, but its computational complexity is much higher compared to SRTES.This aspect must be taken into account when real time processing is a system requirement.

Impact of errors in the atmospheric correction
Figures 15 and 16 show the rmse and the mean error on the emissivity and temperature retrieval, respectively, in case of errors on the values of the atmospheric parameters.Since water vapour is the gas with the highest interaction on the radiant flux in the TIR, we simulate uncertainties on it applying a scaling factor on the given water vapour column density.As in paragraph 4.2, since the performances are quite similar for the tar paper, asphalt, concrete-cobblestone, and graveler, we present the mean rmse values and temperature error, averaged among the four materials.
Figure 15 shows that, for the four materials, the reconstruction of the shape of the emissivities is rather insensitive to small errors in the atmospheric parameters estimation.Though for higher scaling factors the performance degradation is acceptable, an underestimation of the water vapour column leads to high errors, especially on the retrieval of the temperature in the case of LTES and SRTES.As to LTES, as shown in Figure 16, an under/overestimation of the water vapour leads to and under/overestimation of the retrieved temperature.TES-98 and OSTES seem to outperform the other algorithms, being quite insensitive to this kind of uncertainties.Figures 17 and 18 show the rmse and the mean error on the emissivity and temperature retrieval, respectively, for the steel samples.As expected, for the steel samples, in case of small errors, LTES remains the best performing algorithm.When the error increases, the performance quickly degrade, becoming comparable with the ones of the other algorithms.SRTES has a peculiar behaviour: an underestimation of the water vapour, as for the other materials, leads to very poor performance, for the reasons described paragraph 4.2.Instead, if a small overestimation of the water vapour column occurs, the performance improves with respect to the case with an exact estimation.This result may be explained reminding that SRTES, as described in paragraph 2 (step 5), searches the proper values of the emissivity inside small spectral windows.If these peaks are concealed by noise, the iteration leads to wrong estimates, because the comparison between  ̂ () () and its linearization in the same band is coarse.When we overestimate the water vapour column density, we reinforce the peaks in terms of their amplitude.Thus, such overestimation results in a higher SNR in those spectral windows, driving the algorithm to more precise iterations, and, possibly to better performances.TES-98 and OSTES, despite a strong bias (underestimation) on the temperature of about 10 K, seem to be quite insensitive to uncertainties on water vapour concentration, as for the other materials.Therefore, these results suggest that TES-98 and OSTES performances are mostly driven by the noise level.

Real Data
Tables 4 and 5 show the rmse and mean error on the emissivity and temperature retrieval, respectively, for the real data set.  5. mean error on the temperature retrieval (real data set) A general deterioration of the performance, with respect to simulations, was expected, since perfect measurements of the atmospheric parameters cannot be achieved in a real scenario.Furthermore, about the data set, we have not information about sensor calibration (that we assume correct), that could be a source of mistakes, as shown in (Ahlberg, J. 2017).Regarding the steel samples, only the emissivity has been ground-truthed, while the temperature was not measured.The experimental results are in agreement with the ones obtained by simulation.Namely, OSTES and TES-98 perform better than SRTES and LTES when noise sources occur on the signal.LTES and SRTES show poor performances, that are in compliance with the simulations (strongly sensitive to the noise level and to atmospheric uncertainties).Concerning the steel samples, the poor results could be explained by mixed pixels.In fact, as depicted in Figure 5, the steel containers are much smaller than the other objects, so the steel samples may have been partially corrupted by the surrounding materials in that area, such as asphalt and concrete.Of course, this could explain the poor accuracy attained by all the four algorithms over the steel area.However, in general, without any other information, SRTES and LTES cannot be used for the TES assessment.
Figure 19 shows the temperature map of the area, obtained with the OSTES algorithm.The image has not been geo-rectified, since it was not the goal of this work.

CONCLUSIONS
In this paper, a methodology that aims at evaluating the effectiveness of four different TES strategies, over a suburban scenario, has been presented.The methodology is proposed to analyse the algorithms performance during the planning of a remote sensing mission, aimed to discover specific materials of interest in the monitored scenario.Various error sources have been taken into account, such as random noise at the detectors, and uncertainties on the atmospheric parameters retrieval.Preliminary simulations show that the choice of a suitable TES strategy is not an easy task, since the algorithms have different performances on different materials.When Gaussian noise is added to the simulated signal, TES-98 and OSTES overperform SRTES and LTES, in particular for the materials with high emissivity.The algorithms do not show degradation of the performances for the SNRs considered in the simulations, for both the emissivity reconstruction and temperature retrieval.In the latter, the bias is under 1K, basic requirement in many geophysical and surveillance applications.Instead, if the material spectral signature has low values, as for the steel samples, TES-98 and OSTES cannot be used, because are derived assuming a value of the emissivity close to 1.In this case, LTES seems to be the best choice in terms of emissivity and temperature retrieval.
Concerning the performance variability, TES-98 and OSTES seem quite insensitive to both the noise level, and uncertainties on the atmospheric parameters retrieval.For the materials with high emissivity, the algorithms are robust to errors in the water vapour column.Regarding low emissivity materials, again LTES and SRTES outperform TES-98 and OSTES for small atmospheric errors.However, when the uncertainties grow, the performances of TES-98 and OSTES deteriorate rapidly with respect to the other two.Figures 17 and 18 suggest that the performance of TES-98 and LTES could be limited by the SNR and not by the atmospheric errors.
The introduced methodology has been applied to evaluate the TES strategies on a real data set over a suburban scenario.As shown in Tables 4 and 5, the algorithms perform worse in the real case, but TES-98 and OSTES attain the performance predicted during the simulations.SRTES and LTES have poor performances on all the materials (steel samples included), showing an extreme sensibility to the various error sources.The poor results of all the algorithms on the steel samples can be caused by mixed pixels, since the spatial resolution is 1×1 square meters, and is comparable with the size of the steel area.Future works will investigate in more depth the performance of the SRTES and LTES, because they represent the most effective strategies in surveillance applications, where low emissivity objects (e.g.metal objects) have to be detected in a scenario characterized by high emissivity materials.Furthermore, the proposed methodology will be applied on a more recently data set, evaluating also other sources of error and uncertainty on the signal.

Figure 1 .
Figure 1. Outline of the proposed methodologyStep 1 -Materials spectral emissivities from spectral Libraries During the mission planning, several materials of potential interest are chosen from the monitored scenario.In this first step, we should recover the emissivity spectra representative of such materials.For this purpose, we have set up a data base gathering spectral emissivities of many materials.The spectra have been collected from public spectral libraries, such as ASTER Library(Baldridge et al., 2009), MODIS Library (Wan  et al., 2005), and Urban Micromet -University of Reading Library(Kotthaus et al., 2014).
sampling interval;   = number of spectral channels.

Figure 4 .
Figure 4. Example of the relation between   (  ) and (  ) (asphalt sample -T = 311.95K) For this purpose, we refer to a data set made available in the framework of the "Detection in Urban scenario using Combined Airborne imaging Sensors" (DUCAS project)(Cubero-Castan, M. et al., 2015).DUCAS project is conducted under the framework of the European Defence Agency (EDA) by a collaboration among research institutes from different European countries: Belgium, France, Germany, Italy, The Netherlands, Norway, and Sweden.The DUCAS campaign took place in Zeebruges, Belgium, in June 2011.During the measurement campaign, various materials were characterized in a cal/val site, measuring their spectral emissivity and surface temperature.The cal/val site is shown in Figure2, with the position of some ground-truthed materials.The groundtruthed materials used in this work are: tar paper (333.65 K); asphalt (323.15K); concrete-cobblestone (314.15K); graveler (321.15K); and steel (no available information about temperature; for the simulation, we will suppose a temperature of 308.15 K).The site was monitored with the TASI LWIR spectral imaging sensor.The sensor was flown at an altitude of 850 m, for a spatial resolution of 1×1 square meters.In this work, we will use a thermal hyperspectral image, acquired around midday (atmosphere temperature: ~ 32° C).

Figure 6 .
Figure 6.Samples of tar paper.Bold red the average signature.

Figure 7 .
Figure 7.Samples of asphalt.Bold red the average signature.

Figure 8 .
Figure 8.Samples of concrete-cobblestone. Bold red the average signature.

Figure 9 .
Figure 9.Samples of graveler.Bold red the average signature.

Figure 10 .
Figure 10.Samples of steel.Bold red the average signature.

Figure 16 .
Figure 16.Average temperature mean error for different scaling factors of the water vapour column (tar paper, asphalt, concretecobblestone, graveler).

Figure 17 .
Figure 17.Average emissivity rmse for different scaling factors of the water vapour column (steel).

Figure 18 .
Figure 18.Average temperature mean error for different scaling factors of the water vapour column (steel).

Table 1 .
Table 1 summarizes the parameters: Parameters used in the simulations of the scenario

Table 2 .
rmse on the emissivity retrieval.In red the lower rmse for the given material.
The indexes have been calculated (after a visual inspection) for 30 tar paper samples, 28 asphalt samples, 66 concrete samples, 35 graveler samples, and 10 steel samples.