COMBINATION OF THE PHOTOGRAMMETRIC AND MICROWAVE REMOTE SENSING FOR CULTURAL HERITAGE DOCUMENTATION AND PRESERVATION-PRELIMINARY RESULTS

Nowadays, cultural heritage is an integral part of modern societies and it is necessary to preserve the tangible and intangible evidences of the past. Cultural heritage objects and sites are being devastated by natural causes and human acts. There are many existing techniques for structural health monitoring, but in the cultural heritage area, there is a need for a non-destructive measurements. The main advantages and drawbacks of some of these technologies have been presented for both laser scanner and photogrammetry techniques. The aim of this article is to present the integration of existing methods for spatial documentation (classical surveying, laser scanner, photogrammetry) and structures health monitoring using electromagnetic spectroscopy. The multi-sensor platform was developed to characterise and analyse various building materials such as marble, sandstone and bricks. The spatial techniques were used for generation of architectural documentation and geolocalisation of the multi-sensor platform. The preliminary results demonstrated that the proposed technology enables commonly used image and rage-based surface measurements as well as measurements through the entire structure for more detailed analysis. * Corresponding author


INTRODUCTION
Cultural heritage is an evidence of the past that is an integral part of modern societies. It is created by people and societies, constituting a priceless treasure for the current and future generations (Stylianidis, 2019). In order to understand the past and the future, it is necessary to document and preserve the tangible and intangible evidence of the past. Unfortunately, nowadays, cultural heritage objects and sites are devastated by natural causes such as heavy rains, floods, fires, earthquakes and human acts.
It is important to highlight that documentation in the form of the point clouds, 3D models, images and vector drawings is an essential and irreplaceable part of the preservation and should be a high priority (Stylianidis, 2019). When the monumental objects are analysed, the geospatial technologies such as photogrammetry, remote sensing and graphical information systems (GIS) are widely used. In order to perform the architectural documentation and visualisation of cultural object and sites, close-range photogrammetric methods based on the active (range-based) and passive (image-based) method are used (Abbate et al., 2019;Arif and Essa, 2017;Cipriani et al., 2019;Grussenmeyer and Yasmine, 2004;Hatzopoulos et al., 2017;Heras et al., 2019;Markiewicz et al., 2017;Remondino and Elhakim, 2006). Selection of the appropriate method for 3D shape reconstruction and structural health monitoring depends on many factors and requirements of the final documentation (Gonizzi Barsanti et al., 2013).
Exact shape reconstruction of cultural heritage objects is the source of information not only about the 3D shape of the object, but also about the features, which cannot be seen with the naked eye such as cracks inside the object, moisture, etc. For those purposes, electromagnetic sensors with different wavelength are used. Analyses based on multispectral and hyperspectral imagery captured by both passive and active sensors are applied in work regarding historical artefacts of many kind including huge monuments and buildings (Del Pozo et al., 2016;Hemmleb et al., 2006;Lerma et al., 2010), sculptures (Catelli et al., 2018;Hedeaard et al., 2019), paintings and wall paintings (Bonifazi et al., 2019;Cosentino, 2014;Deborah, 2013;MacDonald et al., 2017;Rosi et al., 2013) and parchment (Macdonald et al., 2013).
Spectral data are commonly associated with 2D products such as orthoimages while studying 2D objects like paintings or walls whereas 3D modelling relates to 3D objects (Chane et al., 2013). However, this approach limits the scope of information possible to obtain. To get a comprehensive view of an object it is recommended to integrate spectral information with the object's geometry. Finding optimal way to integrate multispectral data with geometric information was a subject of several studies (Buckley et al., 2012;Chane et al., 2013), but nevertheless this issue is still remaining. Nowadays, terrestrial laser scanning becomes one of the most important technique used for 3D shape reconstruction. The unquestionable advantage of this technique is high measurement accuracy, possibility of acquiring billons of points, level of automation and laser-beam reflectance intensity, which might be used for object properties investigation (Buckley et al., 2012). The main advantages of this method is possibility to measured untextured areas in the short time. However, the main drawback is the problem of measuring the reflective objects, which can effect the quality of the point cloud or completeness of the measured object. The laser beam is fully absorbed by elements of too-low reflectivity; this results in data gaps within the point cloud. In the case of high-reflectivity surfaces, the reflected laser beam does not reach the detector and therefore the locations of points are incorrectly determined (Markiewicz and Zawieska, 2014).
Modern image-based close range photogrammetry is based on the Structure-from-Motion (SfM) and Multi-View Stereo (MVS) approach, which allows for digital reconstruction of the object with the required accuracy and provides a high level of details (Bianco et al., 2018;Buckley et al., 2012;Grussenmeyer et al., 2012;Moussa, 2014;Van Genchten, 2008). The main advantage of this method is relatively low cost of data capturing, but the main disadvantage of this method is a lack of automatic completion, occlusions, blank or poorly textured areas, repetitive structures etc.
It should be highlighted, that for mostly used techniques i.e. image-based and terrestrial laser scanning, visible and nearinfrared wavelength, it is impossible to analyse the structural health inside the investigated object (Markiewicz and Zawieska, 2015;Quintero et al., 2008).
The aim of this paper is to present the novel sensor approach based on the RGB camera, TLS and electromagnetic sensor to demonstrate the potential of penetrating though reflective objects(marble) at the Main Hall of the Warsaw University of Technology.

THE CAPABILITIES AND LIMITATIONS OF THE ELCTROMAGNETIC SENSOR
Electromagnetic spectroscopy has gained the interest in recent years as a promising structural health monitoring (SHM) method in various fields, including offshore, marine and civil engineering. Many researchers investigated electromagnetic sensors as an inspection tool for structures in order to detect structural damages and to assess the conditions of material, which plays an important role in maintenance process. Moisture content is one of the elements, which requires continuous monitoring as structures undergoes rapid deterioration when exposed to aggressive environments and unpredictable weather changes. The electromagnetic sensor for monitoring the excess of the moisture content has been successfully demonstrated as a contactless technique (Kot et al., 2017(Kot et al., , 2016 as well as embedded system for long term monitoring (Teng et al., 2019). In addition, other common structural defects were investigated such as crack development within concrete beams . The use of electromagnetic techniques can be also found in the field of cultural heritage where the remote sensing is highly focused on detecting and identifying objects as well as exploring their features. One of the techniques is ground penetration radar (GPR), which provides architectural framework of a building to plan conservation, restoration or structural monitoring (Tobiasz et al., 2019).
One of the advantage of the electromagnetic waves is a possibility to travel through different medium. Every (building) material has its own dielectric properties, just as it has mechanical and thermal conductivity properties. The important EM properties are the permittivity (or dielectric constant), electric conductivity and magnetic permeability. These parameters are needed to determine reflection and transmission loss through a wall (Vizi and Vandenbosch, 2016). This phenomenon allows investigating the structural health of measured objects not only as a surface measurement, but also across the entire measured sample. The microwave remote sensor also offers non-destructive, contactless measurement performed in real-time, which is of a major interest in the field of cultural heritage, where the physical interaction with measured object is not permitted.

EXPERIMENTAL WORK
The proposed methodology of using non-destructive structural health monitoring based on the integration of the image and range-based method with microwave electromagnetic remote sensing technique was divided in to 6 main steps: (1) Acquiring the TLS and photogrammetric data for intensity and RGB orthoimage generation with depth map, (2) analysis of the RGB and intensity orthoimage in order to detect differences in intensity or RGB (defined the XYZ coordinates), (3) define the possible monumental object desecration, (4) based on the marked control points and surveying technique define the electromagnetic remote sensing devices in the 3D space, (5) EC measurement and (6) analyse the sensor response.
To carry out the experimental work with multi-sensor measurements, four methods were investigated: the classical surveying with total station Leica 1202, the close-range terrestrial scanner Z+F 5006h, close-range images with the Canon 5D Mark II and developed electromagnetic (remote sensing) sensor. In order to analyse possibilities and limitations of the proposed sensor platform, three test sites were documented ( Figure 1).
In order to determine the geodetic/photogrammetric network, the measurement were performed using the Leica 1202 total station that allows measurement with 2"angels accuracy and reflector distance measurement of 2 mm + 2 ppm. The multiple angular and linear observations from several total station positions have be conducted with RMSE 2 mm in order to obtain the high accuracy measurements.
For photogrammetric documentation, the close-range images as well as terrestrial laser scanning data were used. The point clouds from TLS were obtained with resolution 6 mm per 10 meters from 2mrange. For image processing the combination of Structure-from-Motion and MultiView Stereo implemented in Agisoft Metashape Pro approach were used. In Table 1 the statistics are shown.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) The electromagnetic sensor has been designed to be able to carry out numerous measurements without being concerned with its displacement. Sensor parameters such as Antenna angle, distance to the measured object has been recorded to improve the accuracy of the experimental results. Designed sensor operates in the frequency range between 2GHz and 12GHz. The electromagnetic measurements were provided by the S-Parameter, namely S 21 (signal reflected from the measured specimen) from the Vector Network Analyser (VNA) (Rohde & Schwarz). The data was captured via a bespoke LabVIEW program on a laptop connected to the VNA. All the results were recorded to verify if the microwave remote sensor could penetrate through semi-transparent materials namely marble and differentiate various building materials. Figure 1 demonstrate the experimental setup for microwave remote sensing for three different building materials namely marble, sandstone and plastered brick wall.
I II III Figure 1. Experimental setup for the remote microwave sensor.

RESULTS
The point cloud obtained from processed RGB images with oriented EM sensor (based on the classical surveying measurements) is presented in Figure 2. The microwave measurements were repeated three times for each measured object and the response then was averaged and analysed in order to determine the penetration through the marble specimen. The averaged data with and without aluminium sheet placed behind the marble specimen is presented in Figure3. The results show that reflected signal is greater when the aluminium sheet is placed behind the sample. This is due to the dielectric properties of the metal, which reflects the EM spectrum, i.e. blocking EM signal from penetrating further. In addition, this demonstrated that the proposed sensor is capable of penetrating though marble specimen, which is currently a major drawback of mostly used techniques in the field of cultural heritage.  Figure 4 demonstrates the unique EM response for three different building materials namely marble, sandstone and plastered brick wall. It can be seen, from the Figure 4,that there is a significant difference between marble and other tested materials, i.e. more EM energy was reflected back from the marble specimen. The significant changes occurred in the frequency range of 2GHz to 4 GHz with approximately 10dB amplitude change.  The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition)

CONCLUSION
The preliminary results shows that the designed multi-sensor platform allows for full investigation and correct measurement of different objects and materials. The combination of rangebased TLS techniques, image-based techniques (close-range photogrammetry) with electromagnetic microwave remote sensing devices, eliminate the problem of incorrect distance measurement on marble and similar to marble surfaces. Integration of those sensors into one platform allows not only for surface analysis, but also through the entire architectural object. Presented results from the EM sensor shows that different materials can be characterised. Integration of EM data with RGB orthoimages, vector drawings and 3D models can increase the possibility of automatic detection of materials and its conditions for documentation of cultural heritage objects. Authors will further investigate other commonly used materials and their interaction with proposed sensor platform.