USAGE OF PHOTOGRAMMETRIC PROCESSING OF THERMAL IMAGES FOR CIVIL ENGINEERS

In climate conditions of the Czech Republic is getting more important to carry out thermal inspection during documentation and inventory of buildings. Capturing and photogrammetric processing of thermal images requires special photogrammetric approaches especially because of low thermal image resolution. Part of this paper is focused on photogrammetric processing of thermal images using different methods with different 2D and 3D results which help civil engineers and architects better interpretation of thermal imaging. Those results are presented on chosen test objects. Photogrammetric results are also analysed in terms of geometric accuracy. The importance of integration of 2D and 3D results to GIS and BIM for facility management is discussed in final part of the paper.


INTRODUCTION
Thermal imaging is a suitable tool for inspection of for example large historical and administrative buildings.In the case of the Czech Republic, most of the government administrative buildings are situated in the historical part of Prague and those buildings are under strict supervision of cultural heritage bureaus.Every reconstruction of those buildings must be thoroughly substantiated.Thermal inspection of building surfaces allows better argumentation about the necessity of reconstruction.Thermal inspection is usually carried out by calibrated thermal cameras.During thermal imaging is very important to consider many factors which affect the accuracy of the result.Those factors are mainly different emissivity of materials on an object, thermal reflection of atmosphere and other surrounding objects.Every thermal imaging should be controlled by touch thermometer.Thermal images have usually much lower resolution than classic images captured by RGB cameras but still can be photogrammetrically processed while adhering to the proper methodologies.In this paper, the methodology how to capture close range thermal images and how to process them to valuable 2.5D (X, Y, T) or 3.5D (X, Y, Z, T) results is presented.Geometric accuracy or internal orientation of processing will be tested and discussed.The results allow civil engineers better interpretation and decision making than using an only simple thermal image.

Thermal images in photogrammetry
Every object what has a higher temperature then absolute zero emits thermal radiation.If the radiation is captured by a gauge there is a possibility to convert the thermal radiation to the temperature in Celsius degrees.Contactless temperature measurement is an important part of fields such as remote sensing, civil engineering or electrotechnical engineering.Thermal imaging using a thermal camera is a sophisticated work and should be done by a well-trained worker.The interpretation of results is very often difficult because of the many factors which influence measurement.There is a necessity of knowledge about material and characteristic of a measured object.Thermal * Corresponding author imaging is based on knowledge about electromagnetic radiation, * black body objects and real objects, emissivity, and characteristic of a measured object.Compared with RGB images captured by digital cameras, thermal images have a much lower resolution.The resolution of thermal cameras is usually between 160x120 pix to 640x480 pix.Simultaneously, using thermal imaging cameras is not possible to capture some visible features on objects and corners, edges and other important features are captured only if there is a difference in temperature.There are many adverse effects such as reflected temperature from surrounding objects, a different emissivity of each material on the single thermal image, air temperature and atmosphere permeability.Because of those influences, there is a necessity to choose particular methods during thermal images capturing and processing.For photogrammetric purposes, thermal images should be captured with very high overlap.Together with thermal images, RGB images are usually captured as well.For photogrammetric processing of thermal images is possible to use the SfM method.If the images have sufficient overlap and are taken from a greater distance (e.g.aerial imaging) is possible the images orient using SfM method and create a 3D model from the oriented images (Pech, 2013).Also, close range thermal images are possible to orient using SfM method (Scaioni, 2017).For thermal image, orientation is possible to use aerotriangulation as well.(Hartmann, 2012).A much more frequent approach is to process thermal images together with RGB images (Akcay, 2016) Thermal and RGB image is possible to merge and process together using detected corresponding features on the image and transform one image to another in one pair using selected type of transformation (Hoegner, 2016).

Thermal camera imaging for building inspection
For thermal inspection of buildings are mostly used thermal cameras with microbolometer sensor (spectral range 8 µm -14 µm), with image resolution up to 640x480 pixels and with thermal accuracy ± 2 °C.Thermal imaging of the surface of a building should be carried out during much lower air temperatures than the temperature inside of the building to achieve the best conditions to detect thermal leaks.Before imaging should be measured air temperature and air humidity.There are many factors which very affect the accuracy of thermal imaging.Those factors are thermal radiation of atmosphere and other surrounding objects, a different emissivity of different materials on an object and thermal reflection of the sky and other objects.Because of those factors, check measuring with touch thermometers should be carried out.

Flir E95
In this paper, for thermal imaging was used thermal imaging camera Flir E95 with uncooled microbolometer sensor and following parameters.

2.5D THERMAL MODELS
This method is usable for general building facades inspection.Axis of imaging should be perpendicular to the facade and thermal images should be captured with a partial overlap.In the case of high facades, images should be captured even from higher positions and RPAS could be used.Then, thermal images are transformed to 2D CAD drawing of facade or to geodetically measured points.Then a mosaicing is made and is created the 2.5D model (X, Y, T).This method should not be used for facades with a large depth but is very suitable for more less planar facades.This result helps to quickly get a basic idea about thermal leaks on an extensive surface of a building.Through the model is possible to create thermal profiles or even automatically detect thermal leaks.Because of the low resolution of thermal images, the model usually does not have high geometric accuracy.For a better understanding of result is advisable to use CAD drawing together with the 2.5D model.
Figure 2. 2.5D model of building facade where is possible to detect certain thermal leaks.

3.5D THERMAL MODELS
In this paper is presented methodology of creating 3.5D thermal models where information about the location is enriched by an information about temperature (X, Y, Z, T).That can help the civil engineers to better understand and locate potential thermal leaks on a construction.For presenting the methodology, an experiment of photogrammetric processing of thermal data has been carried out.Thermal images of objects have been captured using thermal imaging camera Flir E95 and these images were processed using the photogrammetric method, specifically structure from motion (SfM) method.The geometric accuracy of the results was analyzed.Three locations have been picked for an experiment.It is a part of building façade from the exterior (hereinafter referred to as Exterior), part of the construction from the interior (Interior) and a floor on a terrace (Floor) which is made from two different materialswood and stone.

Photogrammetric targeting and thermal imaging
First, the photogrammetric targeting of ground control points (GCPs) was made.Circular photogrammetric targets were created using a sheet of paper and aluminum foil.The aluminum foil has different emissivity (0,04) compared to the usual building materials such as plaster (0,95), concrete (0,97), PVC (0,90) or wood (0,98).During imaging, the camera emissivity parameter has been set to the universal value 0,90 suitable for thermal imaging of buildings so there was a great chance to identify aluminum targets on the thermal image (the aluminum targets on the image will have wrong temperature).The center of target was marked as a hole.The size of the diameter of the targets was made to correspond 10 -20 pix on the image.For each object were made a sufficient number of targets to have enough welldistributed GCPs on the object and two check points per object to check the geometrical accuracy of results.
Figure 4. Aluminium targets on RGB image.
Before capturing thermal images, the right parameters were set to the thermal camera (emissivity, ambient temperature, humidity, distance from object).Then around 20 thermal images for each object were captured by the thermal camera in the scheme appropriate for SfM processing and with very high overlap and with GSD lower than 0,01m.

Photogrammetric processing of thermal images
Thermal images were exported from Flir software to a simple raster where the value in each pixel is the measured temperature.
For photogrammetric processing, each raster must be converted to the RGB image but there should be kept also the information about temperature.To achieve that, the equation for stretching histogram has been used.The Histogram of temperature of each image was checked and minimum and maximum temperature value for the equation were set.This equation stretches the histogram from 0 to 255.It has to be noted that same f_min and f_max values of the equation must be used same for every image of one image set and should be set the way to have an appropriate contrast of resulted images to be possible to detect many features on the image.If a temperature of an object is very different at each part of an image (e.g.heater where temperature of the heater is very high and temperature of the wall behind is much colder) this workflow is not suitable.But for inspecting building surfaces from interior and exterior and searching for thermal leaks is possible to achieve appropriate results using this methodology.Then RGB image from each image was created using the same values for every R, G and B components and was exported to TIFF format.

𝑔(𝑥
Figure 5. Left image of window is suitable for photogrammetric processing, right image of heater has lack of features and is not suitable for photogrammetric processing. Then set of pre-processed TIFF images was imported to Agisoft Photoscan and was set focal length of the used camera (10 mm) and pixel size of the sensor (0,017 mm).Co-registration of images was made, and photogrammetric targets were marked.Then, bundle adjustment was calculated.Coordinates of GCPs were imported and the model was georeferenced.According to the tables, almost all images of each set were oriented.Total RMS values after georeferencing approximately correspond to average GSD (in case of Interior) or are lower (Exterior and Floor).For each object, two check points were measured to check the accuracy of georeferencing.Total RMS at object Exterior is low compared to its GSD (0,14 x GSD).Total RMS at Floor object and Interior object exceeds their GSD (Floor -1,61 x GSD and Interior -1,43 x GSD).Also, internal orientation camera was checked for each group.Even though the focus of a camera was stabilized during capturing the images of all three sets, the resulted values of interior parameters differ.Figure 6.Resulted radial distortion from image processing.

Results
After image co-registration, georeferencing, accuracy checking and internal orientation checking, the results were exported.Dense point cloud was created in Agisoft Photoscan and exported.The resulted file has an information about the location of each point (X, Y, Z ) and also a color (R, G, B).Using inverse equation ( 1) is possible to convert RGB values back to temperature value and visualize it using Cloud Compare software (Figure 10.)The resulted point cloud is enriched by the temperature value and contains information about location and temperature (X, Y, Z, T).Other results could be textured mesh or orthophoto.
Figure 10.Visualized dense point cloud enriched by temperature in Cloud Compare.

CONCLUSION
Thermal inspection of buildings is an important part of building management and also can help civil engineers to understand the thermal characteristic of a building.Thermal inspection is usually carried out using thermal imaging cameras with very low resolution (compared to the RGB cameras) and thermal imaging should be provided by the experienced worker because the measurement is influenced by many mentioned factors.Photogrammetric processing of thermal images offers to civil engineers to locate temperature data in space.One of the results could be a simple 2.5D model (X, Y, T) which is suitable for building surface inspection and offers to civil engineers general inspection and thermal leak detection on facades.More exacting is to create a 3.5D model (X, Y, Z, T) which requires knowledge of basic photogrammetric principles.In this paper is presented methodology of thermal imaging for structure from motion processing on three different objects (Exterior, Floor, Interior).
After processing, the geometrical accuracy of camera orientation is tested on each object and the resulted internal orientation parameters are compared.The internal orientation parameters of each set of images (Exterior, Floor, Interior) differ and radial distortion is very low and is probably automatically corrected by the camera.This detection should be tested in further research.
The result of the photogrammetric processing of thermal images is point cloud enriched by temperature value.The point cloud could be visualized in subsequent software.Due to this result, it is possible to locate thermal leaks and other thermal characteristics on the object.The geometrical accuracy of dense point cloud should be also tested in further research.Both 2.5D and 3.5D models are stored in very common formats and are also very easy to import them to the many GIS, BIM systems and to building management and facility management software.

Table 2 .
Results of co-registration.

Table 3 .
Ground control points -RMS values.

Table 5 .
Resulted focal length at Exterior object differs from the focal length at Interior object by 0,01 mm but from the focal length at Floor object by 0,08 mm.Differences are also at principle point values.Principle point differs by 13 pixels in one axis.Values at radial distortion are very low.The thermal camera probably has some internal correction.Resulted values of radial distortion do not exceed 1,20 pixel.According to a graph, a trend of radial distortion is more less same at Floor and Exterior object but very differs from Interior object.These facts about resulted interior orientation should be tested in a future on another experiment.Resulted internal orientation parameters from processing.