HIDDEN GRAFFITI IDENTIFICATION ON MARBLE SURFACES THROUGH PHOTOGRAMMETRY AND REMOTE SENSING TECHNIQUES

: Nowadays, considering the various methodological approaches used by the scientific community to study graffiti, there is a need for the improvement of the documentation and analysis workflow that must be addressed. Historic graffiti are recognized by scholars of different disciplines as a relevant and genuine written source, able to provide an insight into the everyday life of the past. Many studies have indeed flourished during the last decades stating the relevance of graffiti as a multidisciplinary source of information and showing, at the same time, the fragmented research scenario due to the lack of reliable and standardized methodologies. Currently, the scientific community involved in the study of graffiti has largely benefited from the use of digital technologies. However, most of the efforts and research projects were focused on the analysis of graffiti immediately visible on the surface or graffiti the position of which was easily identifiable due to the depth and width of the engraving. The proposed study is intended to tackle the documentation pipeline a step earlier than the documentation itself when the graffiti are yet to be identified. The San Marco Basilica in Venice is used as a unique case study for the analysis of non-directly visible-to-human-eye graffiti in a rather challenging environment.


Traditional Graffiti Documentation Methodologies
The main issues in reproducing graffiti are represented by the light scratches composing their ductus, which is difficult to recognize or not visible at all using traditional photographic techniques. Usually, graffiti have been mostly documented through handmade copy. This method is easy to apply since it uses a transparent sheet, a marker and a flashlight. The surveyor reproduces the graffiti by placing the transparent medium on the surface and copying with a marker the visible lines. Often the help of a flashlight is required to better identify the scratches. This is the simplest documentation workflow, but scholars are now aware of its limitations. First of all, this approach is biased by the researcher's ability and knowledge which might lead to a subjective reproduction. It has been verified that different researchers produce copies of the same artefact differing in more or less relevant details and features (Demesticha et al., 2017). Furthermore, during the copy, the human perception tends to select the lines to reproduce, affecting the objectivity of the documentation itself. Additionally criticisms about the mechanical copy concern: (i) the spatial distribution of graffiti in a large area, documented only through the use of transparent, can be lost, together with the mutual relations between them; (ii) the handmade material copy requires direct contact between the transparent and the surface, a factor that sometimes may affect the support conservation conditions; (iii) at a later stage, the copy made on transparent is challenging to be implemented and transferred into other systems, both printed and digital. (Horn et al., 2018). * Corresponding author

Related Works
In the last decade, thanks to the advancement in the digital domain, few techniques have been used to improve the documentation and analysis of graffiti: Reflectance Transformation Imaging (RTI) and Photogrammetry among all. Different studies have been indeed published analysing the pros and cons of each methodology.
In (Mudge et al., 2006;Mudge et al., 2012), the authors presented a study concerning the use of RTI for the documentation of 3D rock art possessing an extensive range of sizes, shapes, and environmental contexts. Polynomial Texture Map (PTM), one of the chief tools of RTI, has been used as recording technology at Roughting Linn rock art site (UK) by (Duffy 2010). The author examined the added value of incorporating PTM into the existing record of the rock carvings.
RTI was used for the documentation of graffiti in the Nile Valley (Frood and Howley et al., 2014). In this scenario, graffiti were realized on stone blocks often as a palimpsest, plastered, and sometimes possibly erased.
For the Musawwarat Graffiti Project (Kleinitz, 2014), the digital recording strategy, including RTI, aimed at the detailed graphic and descriptive documentation of the graffiti in their exact location, taking into account their placement, and motivations of graffiti making.
In (Dhoop et al., 2016), ship graffiti were recorded using RTI in the medieval church of St. Thomas in Winchelsea (UK). Eleven possible objects were identified and recorded, including a variety of ship types.
The application of RTI to ancient graffiti from Herculaneum (Italy) is discussed in (DiBiasie-Sammons, 2018 Today digital photogrammetry can be a feasible and flexible solution, and a quasi-standard procedure may be found to suggest general best practices for systematic 3D surveys. The scientific community studying graffiti has hence started to benefit from this technique largely. Image-based modelling coupled with RTI, Infrared Photography (IR) and X-Ray Fluorescence (XRF) for the study of two mural paintings and their historical graffiti located in the catacombs of San Giovanni in Syracuse (Italy) were used by (Cosentino et al., 2015). The results of the infrared RTI were particularly powerful in their ability to document graffiti on deteriorated surfaces painted with earth pigments.
In (Porter et al., 2016), RTI and photogrammetry have been compared for the analysis of art object from the Palaeolithic site of Solutré (Burgundy, France). The authors concluded that these methods can be seen as complementary and that the resulting products may be used for several purposes, including the creation of stylistic analysis and the dissemination of digital object.
The use of digital tools to document ancient graffiti is described in (Valente et al., 2019). Due to the high level of detail required for the documentation, the authors highlighted that despite the additional efforts required to set the image acquisition and photogrammetric pipeline, the final results are a relevant improvement of traditional photographic documentation of graffiti.

Paper Objectives and Contribution
All the examples, described above in Section 1.2, refer to graffiti where (i) the spatial location is known, and (ii) the readability is clear enough at naked-eye or with the use of raking light. This study intends to tackle those scenarios where, due to the almost invisible depth of the incisions, the graffiti are not visible and are impossible to locate at first glance. A combination of image-based modelling and remote sensing techniques is used and described below in Section 3.
In the San Marco Basilica, different factors prevent direct graffiti identification. The traditional techniques used to detect them, such as UV light or oblique light, although being helpful are indeed not conclusive.
The inner space of the Basilica is wholly covered and decorated with juxtaposed marble slabs. Restoration works and routine cleaning have removed the natural patina and critically reduced the depth and visibility of graffiti, now faded and almost disappeared.
A second issue in detecting and documenting graffiti on marble slabs is that the surface is extremely reflective. Light sources, though, must be adequately and correctly located and calibrated to avoid further disturb, and allow better visibility.
Moreover, in the marble revetments, the veins of the stone create a sort of continuous drape, as is still seen today in the Justinian churches of S. Sophia in Constantinople and of S. Vitale in Ravenna. This polychromy produces a non-neutral visual background where the shades of colours can be easily confused with scratched lines. Due to these reasons, the knowledge of such an amount, in number and quality, of graffiti inside the Basilica was not known until the present survey.

The San Marco Basilica of Venice
The San Marco Basilica is the principal monument of the unique city of Venice. It represents and summarises the history and the vocation of the city, settled between the land and the sea, between East and West. Together with the Doge's palace, it embodies the political and religious power of the Republic of Venice. The building was erected during the 10th century as the Doge's private chapel. It was gradually expanded and decorated from the 12th century thanks to the income derived from Venetian commercial activities in the eastern Mediterranean and the crusades (mainly the fourth, guided by Enrico Dandolo, the Doge himself). Spolia from Constantinople and the Greek territories reached Venice to decorate and embellish the Basilica and many other private buildings in the city. The result is the magnificent monument still visible today. The external is a rich marble casket realised with precious architectural elements that constitute the background of the Piazza, where the main political and social events of the city took place. Once inside the visitor is immersed in an astonishing space, created by marble revetments in the lower part and golden mosaics covering the five domes and the vaults. This overwhelming environment has always inspired the visitors, in the past as well as today, combining the physical beauty of the monument to its religious and political value (Puppi et al., 1999).
Visitors' wonder and devotion have been registered through the centuries with the scratching of graffiti. Few works from the end of the 19th century mention the presence of Armenian graffiti inscription and ship reproduction on the main facade (Zorzi, 1877;Predelli, 1890;Helms, 1975;Ray Martin, 2001). As far as now, few other inscriptions and drawings were known thanks to their deep and well-defined furrow that made them visible. They are preserved mainly in the upper-level women's' gallery and in the external terrace, behind the bronze horses.
Recently, a more detailed survey of the ground level marble decoration, realized by the Andreas Pittas Art Characterization Labs (APAC) of the Cyprus Institute, showed the presence of very faint linear features that, due to their state of preservation, have not been previously identified as graffiti but just as accidental scratches.
In the framework of this pilot case-study, it has been decided to focus the search and documentation efforts in two areas, easily reachable and located in the transept (south-west part) with direct visual access to the apse and altar (Figure 1). The first area is a portion of the wall located next to the entrance of what today is the 'Treasure of the Basilica'. The second one is instead positioned in the lower part of one face of the first pillar on the left, flanking the central nave (Figure 1). These locations are also still enriched by the presence of marble benches. The presence of the latter has been considered as an important element that could foster the making of graffiti by providing the carvers with a comfortable position that could have allowed them to realize, undisturbed, not only simple drawings but also extended portraits, scenes and inscriptions ( Figure 2). In Table 1 the extension of the walls surveyed is described.

Location Dimension Area
South Transept 1.2m X 1 m 1.  Figure 3 summarizes the methodological workflow developed for the first identification and documentation of invisible graffiti. Figure 3. The workflow adopted in the project.

DATA COLLECTION AND PROCESSING
After a photogrammetric survey of the marble decoration and the production of the relative orthophotos where graffiti are suspected to exist (section 3.1), the pipeline consists of (i) computation of edge-detection to identify graffiti (section 3.2); (ii) computation of change-detection to highlight the differences between the pre-processed image and the one created by the edge-detection algorithm (section 3.3).
The orthophotos have been processed using the open-source software Orfeo ToolBox (OTB), a remote sensing image processing library developed by CNES, the French Space Agency, including a fast image viewer, applications callable from command-line, Python or QGIS, and a powerful C++ API. (Grizonnet et al., 2017).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W15, 2019 27th CIPA International Symposium "Documenting the past for a better future", 1-5 September 2019, Ávila, Spain

Photogrammetric Survey
The photogrammetric setup consisted of a Canon 5D Mark IV camera, 30-megapixel full-frame sensor (6 µ pixel size), equipped with a Canon EF 70-200 mm f/4L IS USM lens, two photographic lamps, polarised sheets, and polarizer filter. The camera's autofocus was disabled, and the lens focusing ring and Focal Length were fixed at 200 mm with a piece of insulating tape to avoid accidental changes of interior orientation parameters during the photogrammetric survey.
In 3D dense stereo matching, surface reflections may lead to incorrect measurements and blunders in the resulting dense point cloud. The use of a lens polarised filter, oriented in the same polarisation plane, mainly results in a surface enhancement effect, increasing the saturation and contrast of the image. However, with this configuration, the reflections are not eliminated. In order to overcome the problem of disturbing reflections, such as for the marble slabs of the San Marco Basilica, polarising filters can be mounted both on the camera lens and on the light source(s). Reflections in the images can be then suppressed by crossing the polarising direction of the filters leading to homogeneously illuminated images and better matching results. To achieve this goal, two polarised sheets were placed over the circular lamps, equipped with fluorescent light bulbs, and rotated accordingly until perpendicular or crossed planes of polarisation were reached.
Few studies have been published concerning the use of polarising films for image-based 3D reconstruction (Table 2).
A photogrammetric camera network was a priori planned , following a normal and convergent schema, with the primary aim to guarantee the automated matching of homologous points and an adequate number of intersecting rays (Fraser, 1987). The images were acquired with a mean overlap between two consecutive normal views of about 75-80%. Moreover, convergent images were acquired, yielding convergent angles of 45°.
Given the requirements of sub-millimetre accuracy, a photographic scale of 1:10 was selected. An estimated cameraobject distance of 2 m was set, thus resulting in a mean Ground Sample Distance (GSD) of 0.06 mm. An average base-to-depth ratio (B/D) of 0.5 was computed. Starting from the estimated camera poses and orientation, a dense 3D reconstruction via a pixel-based image matching algorithm was applied. This was performed using the first-level image pyramid, corresponding to the original full image resolution. The derived orthomosaics, radiometrically balanced, and digitally blended so that the seam lines between images are not visible, were finally produced with a pixel size of 0.13 mm (Figure 4). Several Ground Control Points (GCPs), regularly distributed on the area of interest, were measured with a surveying network to scale the model and avoid any deformation due to the planarity of the object digitized.

Edge-Detection
Edge-detection is an image processing technique, mainly used in the remote sensing domain, for finding the boundaries of objects within images. It works by detecting discontinuities in brightness, computing edge features on every pixel of the input image selected channel. In recent years, many approaches have been explored for automatic edge-detection (Roberts, 1965;Canny, 1986;Touzi et al., 1988;Sobel, 1990). For the detection of graffiti in the San Marco Basilica, the Sobel edge detector implemented in OTB has been used. It exploits a pair of 3 x 3 convolution masks, one estimating gradient in the xdirection and the other estimating gradient in y-direction ( Figure  5).
It is anticipated that, due to the intrinsic features of the marble slabs used in San Marco, and characterized by linear veins, the algorithms created several artefacts which were discarded by manual and visual inspections at a later stage. Figure 5. San Marco Basilica, western wall of the southern transept, orthophoto of the Crucifixion scene, edge-detection

Change-Detection
Many techniques have been developed in the remote sensing domain, with the purpose of identifying the differences occurred on the Earth surface over time. This task usually referred to as change-detection, can be approached in a supervised or unsupervised way, being the latter preferred when no training samples or only a little knowledge on the ground is available.
The Multivariate Alteration Detection (MAD) algorithm (Nielsen et al., 1998) was selected in this study with the main aim of highlighting the changes occurred on the analysed surface after the application of the edge-detection step. Since MAD analysis lacks in semantic interpretation, the adoption of a combined procedure can be preferred to support the understanding of changes found by MAD. For this reason, Nielsen proposed to apply the Maximum Autocorrelation Factor (MAF) transformation to the MAD components (Nielsen et al., 2010). The compared images used as input for the MAD/MAF analysis, were a desaturated version of the original RGB orthophoto and the outcome of the edge-detection process. Concerning the change-detection step, no image alignment was performed since the input and output data share the same physical space.

RESULTS
The results of the application of edge-detection and changedetection algorithm are twofold. First, thanks to the computation of the edge-detection process, graffiti invisible at naked-eye have been located and highlighted (Figure 6 and 7). Second, undocumented graffiti, have been identified and located.
Within the field of graffiti studies, the applied techniques represent an original and very efficient digital workflow able to improve scholar's ability to detect, verify and document, using high-resolution images, this very feeble heritage. In normal environmental condition (indoor spaces), indeed, graffiti detection with naked-eye is challenging, and often the traditional methods fail. Moreover, although the application of photogrammetry and RTI have improved visible graffiti documentation, the present contribution brings the research a step forward. Exploiting remote-sensing born algorithms, it has been demonstrated how it is possible not only to enhance visible graffiti but also to locate non visible ones.
Two graffiti represented the most impressive examples: the scene of the crucifixion (Figure 5), and a portrait identified as a Doge's head ( Figure 6).
The crucifixion, due to the extended area of the graffito, is characterized by a high concentration of scratches. Moreover, the graffito is located on a marble section with natural veins that hampered the complete and correct reading of the drawing. In this case, the described digital workflow crucially enhanced the general understanding providing a global view of the scene with relevant details. It was possible to see the peculiarity of the blood spreading from Christ's rib cage, and to identify a second graffito (on the left) reproducing a horse, that is juxtaposed to the crucifixion.
The Doge's head, identified as such thanks to the characteristic corn hat, has a very small dimension which did not allow the identification with the naked-eye. The proposed methodology allowed to recognise it with a high level of definition, recording every detail. Figure 7. San Marco Basilica, pillar, orthophoto, inscription detail, change-detection

CONCLUSIONS
This study proposes a new workflow for the initial identification of graffiti which due to the material properties, the geometrical features of the scratches and the environmental conditions are invisible and not directly perceivable by the human eye.
The described pipeline is mainly based on the application of photogrammetric techniques and remote sensing algorithms not directly developed for the graffiti domain.
From a broader point of view, this project represents a new frontier for the field of study of graffiti since it allows to identify their presence in challenging contexts. It is now possible to perform a more detailed analysis, detecting the presence of such elusive evidence that can help not only to enhance our specific knowledge about graffiti but also connect them to the monument where they are located, its history and use, such as for the Basilica of San Marco in Venice.
RTI, which became a quasi-standard procedure in the analysis and study of graffiti, is not intended to be replaced by the described approach. It must be instead stressed that once invisible graffiti are pinpointed on extended surfaces, RTI can still represent a valuable source of information focusing on specific artefacts, making the two techniques complementary.

FUTURE WORKS
Considering the positive and relevant results obtained in the framework of the presented pilot case-study, future activities will focus on a systematic graffiti documentation campaign. Due to the extension of the San Marco Basilica inner space and pursuing the efforts of time and man-efforts optimization, the next phase of the research will be represented by the implementation of a customized photogrammetric rig which will help in the acquisition of a broader area of the Basilica.
Despite the presence of a-priori documented graffiti, the photogrammetric documentation will indeed regard an extensive band of the marble decoration from the ground level to 1.5-meter height. This approach is expected to allow a more time efficient data collection without reducing the geometric and radiometric accuracy of the outcomes.