PANSHARPENING TECHNIQUES TO DETECT MASS MONUMENT DAMAGING IN IRAQ

The recent mass destructions of monuments in Iraq cannot be monitored with the terrestrial survey methodologies, for obvious reasons of safety. For the same reasons, it’s not advisable the use of classical aerial photogrammetry, so it was obvious to think to the use of multispectral Very High Resolution (VHR) satellite imagery. Nowadays VHR satellite images resolutions are very near airborne photogrammetrical images and usually they are acquired in multispectral mode. The combination of the various bands of the images is called pan-sharpening and it can be carried on using different algorithms and strategies. The correct pansharpening methodology, for a specific image, must be chosen considering the specific multispectral characteristics of the satellite used and the particular application. In this paper a first definition of guidelines for the use of VHR multispectral imagery to detect monument destruction in unsafe area, is reported. The proposed methodology, agreed with UNESCO and soon to be used in Libya for the coastal area, has produced a first report delivered to the Iraqi authorities. Some of the most evident examples are reported to show the possible capabilities of identification of damages using VHR images.


INTRODUCTION
This paper illustrates the study of the use of pansharpening techniques on very high resolution satellite imagery to localize and highlight the damages suffered by monumental and historic sites in Iraq during the recent years. The project, appointed by the General Secretariat of the Italian Ministry of Cultural Heritage and Activities and Tourism, is developed in the frame of an Italian-Iraqi bilateral agreement between the Italian Ministry of Cultural Heritage and Activities and Tourism and the State Board of Antiquities and Heritage of Iraq upon financing of the Italian Ministry of Foreign Affairs and International Cooperation, to study the consequences of terroristic attacks on cultural heritage in some Iraqi areas. Very high resolution satellite images were employed mainly for two reasons. The first is that a field survey could have been very difficult to manage due to the actual problematic situation of safety in those areas. On the other hand, some of the observed damages are so extensive that a first evaluation can be detected efficiently also on the sole base of satellite imagery at submetric resolution acquired before and after the events. For the specific study, they were acquired before and after the critical period of 2010-2015 years, characterized by the main destructions. Before these images are ready to be used for interpretation they have to be processed according one of the several data fusion approaches available. In fact, in the field of the remote sensing, the introduction of high resolution satellite sensors has required the development of specific techniques as * Corresponding author two main kinds of images are usually acquired and available: multispectral and panchromatic. The first group present lower spatial resolution but accurate spectral information, while the second kind present a higher spatial resolution with a wider band acquisition in the "visible" range. Pan-sharpening is a techniques that allows to combine panchromatic and multispectral data to create new multispectral images with the geometric resolution of panchromatic band and colour depth of the multispectral bands (Zhang, 2012). It is always used when thematic information have to be extracted from VHR images in manual or automatic (Baiocchi, 2014) procedures. For this project, different pansharpening methods were tested to obtain, for the specific use and the specific characteristics of the area, the best compromise to highlight the observable changes. Different treatments were needed also because the images were collected from different platforms, choosing for each site the images more suitable for the specific detection. Thus, the sites, chosen by Iraqi technicians, and positioned on paper maps, were identified in the archives satellite images acquired before the start of the destructions in 2010. Then new acquisitions were ordered on the same locations, possibly with same satellite platform, around August 2015. The used platforms were released by the Satellite Imaging Corporation, owner of some specialized satellites (Digital Globe's WorldView-2, WorldView-3, GeoEye-1). This visual check permits to localize the main and more evident destructions occurred in some attentioned Iraqi areas, and after to proceed to a depth and specific studies to define and quantify the damages; but this last is not object of the present article.

MATERIALS AND METHODS
For every single site two images were chosen for the "before" and "after" representation. The choice was aimed to obtain the most similar image in the two moments to be sure that change observed cannot be due to different platform or acquisition modalities. The satellites utilized for this study belongs to a new generation that have improved characteristics if compared with the first VHR satellites like, for example, Ikonos and Quickbird. The first difference is in the number and spectral range of multispectral bands. Thus, it's very easy to find eight or more different bands (Table 1) when in the past some satellites had only panchromatic band (i.e. Eros-A) but, anyway, no more than four multispectral bands were available, usually: Red, Green, Blue and Near-IR. This was a very useful characteristic in this research cause the possible combination of different bands with different algorithms allowed to find ever the better solution for a proper visualization of every single site. Thus, these new platforms show an improved geolocalization procedure so that also without Orthorectification an accuracy of 5 meters is achievable (Tab. 2).
Orhorectification is a procedure that is usually needed to remove all the deformations due to the acquisition process as prospective effects, height displacement, atmospheric corrections , (Toutin, 2011), (Tao, 2001a) and (Tao, 2001b). The orhorectification process need for specialized softwares, knowledge of satellite charcteristics and Ground Control Points (GCP). GCPs have to be surveyed with GPS/GNSS receiver whit specific campaign on the ground. Recently the geolocalziation devices (mainly GPS/INS) were improved so that, satellite and aerial images have good positional accuracy also without user orthorectification; for example, some open access imagery has the accuracy of medium to high scale accuracy (Pulighe, 2016). Obviously, for these specific application this was a very useful property of imagery because it allowed to reach an accuracy of positioning similar to a medium scale maps (i.e. 1:25000 scale) without the need of a ground campaign of survey, obviously very complex in these situations. More in detail, the use of WorldView-3 asks for specific strategies due to its particular characteristics. It was launched by DigitalGlobe during 2014 and is considered the first multi-payload, super-spectral, high-resolution commercial satellite sensor. WV-3 has sun-synchronous orbit and sensors have a swath width of 13.1 km. By operating at an altitude of about 617 km, it has an average revisit time less than 1 day: it daily collects up to 680,000 km 2 . In addition to the "classic" Panchromatic and Multispectral bands, WorldView-3 has 8 short-wave infrared (SWIR) bands and 12 CAVIS ones (useful to provide atmospheric).
Panchromatic and multispectral scenes have a dynamic range of 11 bit (2048 different tones possible) while the radiometric resolution for the SWIR is 14 bit. WV-3 satellite represents an incremental improvement compared to the previous WorldView-2 (also used in this work) because it carries on-board also sensors with multispectral SWIR capability. The SWIR bands are useful in several applications: surface compositional modelling, mapping of rock and soil exposures. More specifically, WorldView-3 provides the following different geometric resolutions on the specific bands: • up to 0.31 m for the panchromatic band; • up to 1.24 m multispectral bands; • up to 3.7 m for the short wave infrared bands; • up to 30 m for the CAVIS bands. These different resolutions results in a more complex strategy to find the most suitable pansharpening methodology.
We had the possibility to test the pansharpening using three different approaches that in literature are between the most widely diffused: Zang; IHS and Brovey. Here, we'll illustrate them shortly, but they are fully documented in literature. The Zang algorithm is widely described in literature (Zang 2002a) and (Zang 2002b), and its equation to fuse panchromatic and multispectral bands can be written as: where MSi is the i-th multispectral band PANorig is the panchromatic band PANsyn = ∑ ᵠ * = Instead, with IHS approach, multispectral images are projected from RGB (Red-Green-Blue) to IHS (Intensity-Hue-Saturation) color space (Zhang, 2004). In the new "color space", Intensity is near to the panchromatic image but not exactly coincident: the difference can be estimated (Tu, 2004) as: where I, can be defined (Tu, 2001) One of the advantages of IHS approach is that it can be easily implemented also in open source GIS environment  Brovey approach, instead, was developed to increase the visualization contrast in the low and high ends of an image's colour range (Saroglu, 2004). In Brovey the fused multispectral images are calculated as:

= *
where MSi is the i-th multispectral band MStot is the combination of multispectral images Rigorous comparisons of the results of different pansharpening strategies can be performed using different approaches, between Universal Image Quality Index (UIQI), (Maglione, 2016), but for this study the most important parameter was the correct photointerpretation by the final users so, the results of the different procedure were compared mainly in terms of operative possibility to distinguish the changes between the "before" and "after" imagery.

RESULTS
For this study, seven complete scenes taken before the events were acquired and five complete scenes of the period after the events were used. The overlap between the seven "before" and five "after" scenes guaranteed the possibility to check for damages on several known sites. All the images were ordered in "bundle" version so the panchromatic and multispectral were available for each acquisition. During the tests, it appeared that Zhang approach was the most versatile for this specific application and so, also if the results of the other approaches were valid, the final visual interpretation have been performed on Zang pansharpened imagery. The destructions were not always easy to detect, in fact, in some cases, the ruins were completely erased and the space left reused. A simple visual observation, in an urban complex texture as the one in which we're working, led to a very little number of identification, so some sites, chosen by Iraqi technicians, and positioned on paper maps, were identified on the acquired Figure 4. Mosque of Profet Nebi Yunus after the events satellite images acquired before the start of the destructions in 2010, then, on the same position on the "after" images the changes were observed. One of the most evident example is the Mosque of Profet Nebi Yunus, that it is detectable on the image before the event (Fig. 3).
In the image, it appears completely intact; the jigsaw appearance of some details (see for example the road side wall going North to South in the left part of Fig.3 compared to Fig.4) is in fact due Figure 5. Mosque and sacred place of al-Nabi Sheet before the events to an interpolation flaws. These flaws seems to have effect not only on the geometry but also on the pansharpening results, in fact some unnatural colours appear in correspondence of the "jigsaw" areas. This is probably due to a geometric misalignment that causes wrong overlapping of the pixels of the different bands.
After the destruction, there is no remain of the mosque and it is not observable any remain or debris of the monument parts; this can suggest that almost a part of all the material was transported elsewhere, probably to limit the possibility of an eventual reconstruction. In fact, observing carefully the image acquired after the destruction (Fig. 4) it is easy to recognize some new tracks on the central part of the top of the image, that were probably used by trucks that moved all the debris obtained after the demolition, somewhere else.
Sometimes, instead, it was easy to detect the ruins that were moved not far from the original position of the monuments as, for example, the case of the Mosque and sacred place of al-Nabi Sheet, see top left of figure 5. The mosque and sacred place were completely destroyed (Fig. 6) but it's easy to detect that at least one part of the ruins, that appears to have the consistency of a thin dust, was unloaded directly on the river bank near here (still Fig. 6) probably using a bulldozer. In addition, in this case it is possible to observe some flaws probably related to the geometric processing of the image and it is easy to observe as the unreal colours defects related to the some geometric flaws. In other cases the teledected images evidenced the presence of concrete on the sites were the historical sites and or buildings were destroyed. The most evident example is the destruction of the complex of the mosque dedicated in 1300 to the Prophet Nabi Jirijs in Ninive. Also in this case it is possible to observe the misallignment flews in the image before the events (Fig. 7) and their effects on mismatching of the different bands on the pansharpened image. After the event (Fig. 8), it is possible to recognize the site only from the shape of the block and from the pattern of the surrounding buildings. In fact, the monument was completely erased, the ruins dumped somewhere else and it seems that a layer of concrete had been placed in the original position to use the space left as a public parking lot. This is not the only example of this reuse of the empty spaces left after the destructions, a big number of them was observed, during the work of the team involved and what sometimes seemed astonishing is the surgical precision in the destruction of a single building maintaining the neighbour construction. In Figure 9 and 10, is in fact possible to observe that the total erasing of two monumental buildings didn't affect at all the contiguous building that seems to remain untouched after the demolition (see the building in the left upper corner of figure 9).

CONCLUSIONS AND FURTHER DEVELOPMNETS
The proposed methodology, agreed with UNESCO (soon to be used in Libya for the coastal area), has produced a first report delivered to the Iraqi authorities. It will support as a documentation to recognize actual state of health of cultural heritage, the damages incurred and understand possible strategy for reconstruction; this is important to organize future restorations in the areas and to understand a possible politics of preservation for cultural heritage on the territory. With the aim to rank the techniques efficiency, for these specific tasks and to delineate procedures for future similar tasks, visual inspections combined with quantitative evaluations were performed to test spectral qualities of the fused images.
The processed images showed, to cite only some of the inspected sites: the destruction of the complex of the mosque dedicated in 1300 to the Prophet Nabi Jirijs and the thirteenth century shrine of Imam Yahya Abu al-Qasim, built by Governor Badr al-Din Lu'lu'. Sometimes the teledected images were used also to try to individuate the final location of the debris of the destruction because it is not clear if they are only destroyed or removed to other locations. Sometimes the use of teledected images evidenced the dumping of the debris near their original positions. In other cases the teledected images evidenced the presence of concrete on the sites were the historical buildings were destroyed. The future development of the project provides for the creation of a joint working group, series of lessons by the Italian part of the team on the use of satellite imagery and for the composition of groups of local technicians to intervene on sites currently destroyed.