REMOTE SURVEY OF TRADITIONAL DWELLINGS USING ADVANCED PHOTOGRAMMETRY INTEGRATED WITH ARCHIVAL DATA: THE CASE OF LISBON

The application of advanced photogrammetry and the integration of cross-sourced maps of historic cityscapes contribute to a thorough understanding of our surrounding, enabling the comparison between present physical layout/use of the built environment and its changes over time. Remote surveys integrated with the analysis of building dossiers, stored in municipal archives, and onsite surveys are carried out in this work for providing insight into a traditional building typology in Portugal, which is so far overlooked in heritage studies. Two examples of humble multi-family dwellings with overhanging timber-framed façades (16th century) are virtually reconstructed using structure-from-Motion (SfM) of selected historic photographs. During a first phase, a photogrammetric reconstruction of the state of the buildings, as conceived originally, was implemented by reconstructing two parts (due to the number of images available for each case study), and then a metric evaluation of the models was conducted comparing past and current state of the buildings. This research shows the importance of integrating distinct tools and databases to foster truly tourists' perception of authenticity in cultural heritage sites and towards a more sensitive heritage management. Potential interactive applications capable of increasing the visibility and impact of these virtual reconstructions, such as GIS data enrichment or digital travel apps, are also briefly discussed. * Corresponding author


INTRODUCTION: HISTORICAL IMAGES FOR EXPLORING HISTORIC CITYSCAPES
One of the most critical issues in historic cityscapes regards to which extent intergenerational communities can identify hidden histories and timeless cultural heritage values. Do we truly know the city we live in? How to critically analyze the changes in historic cityscapes? To address these questions, this research shows the potential of the advanced photogrammetric technique for understanding site characteristic of streetscapes, urban fabric, historic building features, and local construction systems employed over time. In general terms, photogrammetry allows researchers to extract reliable metric information about objects through the 3D virtual modeling. In ever-changing historic cityscapes, the interpretative analysis combined with advanced survey approaches for extracting archival contents is required to unveil multiscale features of the built environment, especially when just few architectural vestiges exist in the real environment. Extending the scope of application, the use of advanced photogrammetry allows to reconstruct single buildings (or architectural ensembles) that were destroyed (or highly modified) over the centuries. The traditional metric survey can be also hampered in post-seismic or hard-to-reach areas or in war zones or if the cost of on-site surveys is prohibitive. In many cases, the use of multiformat archival datasets is one of the most reliable ways to record relevant changes in the external environment and, potentially, it can be used for GIS data enrichment or for visualizing digital apps for tourists/visitors or local communities living in. In this research, it is shown that the proposed remote survey method represents an effective zero-cost tool for in-depth analysis of architectural and construction components. If integrated in freely accessible apps or tools, the results obtained from this analysis can contribute to a truly perception of what is authentic in heritage sites nowadays. Putting this in perspective, this method can inform more sensitive heritage policies. Two traditional overhanging dwellings in Lisbon (16th century), known in Portugal as casa de andares em ressalto, are virtually reconstruct using selected historic photographs. Being both corner buildings in traditional parish and rare example of timber framed buildings in the urban history of pre-1755 Lisbon, the selected case studies are landmarks to be preserved for their cultural value (Gshwend and Lowe, 2015;Stellacci et al., 2016). Being originally inhabited by underrepresented groups (ethnic minorities, low-wage workers) that form the backbone of the Portuguese economy and culture, these buildings also embody a meta-historical value. The proposed research methodology has been implemented in previous virtual 3D reconstructions of demolished (or extremely changed) monuments in Cappadocia, in eastern Anatolia (1925) and in Japan (2013) (Condorelli et al., 2019).

OPEN ISSUES ON HISTORICAL ARCHIVAL IMAGES IN LISBON
Plenty of pictures, audiovisual materials, or project drawings, stored in municipal archives or diverse private repositories, encloses fragments of physical and symbolic territories that should be brought to light. Such type of archival data is an irreplaceable bearer of multi-layered memories since the analysis and processing of each photographed spot can unveil hidden or ignored histories. The diachronic visualization of building metric information using archival content contributes to the knowledge of the destroyed or heavily changed built environment.
In absence of adequate documentation, the analysis and reconstruction of bygone times is possible only using historical images (Bitelli et al., 2007). In orienting and registering historical images, we gain new insight into unknown aspects or perspectives on livelihoods or details that are hardly understandable in an accurate way just using bidimensional images, such as the overhanging structures. As shown by Maiwald et al. (2018), multiple facets can be reconstructed on a reliable resource basis and then explored in a 3D environment. Al Khalil and Grussenmeyer (2019) show the potential of overlapping past and current states of cultural heritage artifacts. The advanced photogrammetric technique offers manifold opportunities, such as the accurate reconstruction of the geometry and, more importantly, the integration of multisourced and multi-layered data about surface quality, colour, structural damage patterns, and other contextual-based information, so long as good quality materials for data processing is available. Nevertheless, archival photographs are rarely used for building metric documentation and 3D reconstruction purposes since camera parameters and film information are often missing.
Focusing the attention to the building typology casa de andares em ressalto, we should underline that it is common in the Northern Portugal, e.g., Porto, Lamego, Guimarães, Vila Real, and Chaves. Due to 1755 earthquake and the demolitions or high alterations of the built tissue over the centuries, rare cases of this typology still exist in Lisbon. A group of forty dwellings belonging to this building typology in the traditional parishes of  Figure 5). In addition to historic visual contents (photographs, postcards, paintings, videos), most of the information about historical buildings comes from municipal dossiers (Processos de Obra). These municipal documents contain a set of information about the history of the whole building or related to some parts (e.g., ground floor). These dossiers include inspection reports and requests with respective municipal decisions that were submitted by the owners (or tenants) of each building to the Council for the approval of the intervention works. Other miscellaneous documents consist of technical reports written by architects and builders with architectural drawings, before and after specific intervention works in the buildings.
In the case analysed in this study, it was possible to compare the 3D reconstruction obtained from historic photographs stored in the Lisbon Archive (AML, Arquivo Municipal de Lisboa), the respective building permission of the Council, and the current state of the building. The historic photographs of this building typology in Lisbon stored in AML generally show the exterior buildings or few details (e.g., balcony). These result in one-of-a-kind prints that depict suggestive scenes of everyday life, such as smoking factory chimneys, rows of clothes hanging, children playing, humble passers in dark suites, elderly people looking at the window, horses with carts, and goods to sell. The iconographic and documentary databases on the case study No.1 is relevant in quality and number of documents. In fact, this traditional dwelling was photographed by seminal photographers in Portugal, such as Joshua Benoliel , José Artur Leitão Bárcia (1873-1945), Armando Maia Serôdio (1907-1978), Ántonio Gárcia Nunes (1878?-1946?), Eduardo Portugal (1900-1958, Alvaro Ferreira da Cunha (1901-1970), Artur Pastor (1922-1999. It was also depicted in other documents, such as historic postcards (Passos, 1997) that are reproductions of the 19th photographs (e.g., Eduardo Portugal), which are originally gum-bichromate and bromoil printings. A set of 14 photographs on case study No.1 was selected for the 3D reconstruction. Unfortunately, the camera shooting points, pointing towards the main façade around the steep slope of S. Jorge Castle, are similar (low overlap).

REMOTE 3D SURVEY USING HISTORICAL IMAGES AND ON-SITE SURVEYS
Data processing of archival visual datasets could be hampered by e.g., low overlap and low resolution of the original format image. These limitations are overcome boosting the photogrammetric pipeline as shown in Figure 6. The first phase is divided into two sub-phases, according to the availability of the historical data, while the second phase is similar in both two case studies. Each sub-phase of the workflow is reported as below.
For the reconstruction of case study No.1, the SfM procedure was implemented using the COLMAP opensource Structurefrom-Motion (SfM) and Multi-View Stereo (MVS) algorithm implementation, developed by ETH Zurich (COLMAP, Johannes L. Schoenberger, 2019). COLMAP SfM sequential processing pipeline is based on i) feature detection and extraction; ii) feature matching and geometric verification; iii) structure and motion reconstruction. Some important radiometric corners in the image, while appearing in other images, may be missing due to the low quality of the initial data. To avoid this limitation, the manual selection of specific feature points was carried out for guarantying a more accurate metric evaluation of the final point cloud. This step allows the selection of those specific points and the extraction of respective 3D coordinates. Figure 6. Workflow for the 3D reconstruction of selected case studies.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France The application of the SfM algorithm used in this study for the virtual reconstruction is shown in Figure 7. The step of manual selection of specific feature is highlighted with a darker grey hatch. Figure 7. COLMAP SfM sequential processing pipeline with the addition of the new step of manual selection of specific feature point As regards case study No.2, the availability of few images was overcome by collecting additional data. In photogrammetry, the reconstruction of an object from a single image is possible when is integrated with additional visual information. Single image processing is usually used for orthophotos and for measuring flat objects. The accuracy that can be achieved when measuring objects depends mainly on the image scale and the ability to distinguish the features to be measured in the image (Luhmann et al., 2020). Therefore, the rectification of the archival photographs was conducted using the software RDF based on a set of geometric principles, such as linearity, parallelism, orthogonality, symmetry, to calculate intrinsic camera parameters, principal point coordinates, and the planar surfaces of the object. These are rectified using vanishing points as detected in the image (Bräuer-Burchardt and Voss, 2002).
The building measurements obtained from the processing of the historical images were compared with the existing building to validate the 3D building model. Unchanged features, such as opening layout, the door-way size, and the nodes of the exterior envelope of the building, were used for carrying out the 3d reconstruction. The validation consists in calculating the residuals between the measured distances. Other metric information is obtained considering that vernacular building proportions were influenced by palm (22cm) (Carita 1994). The exterior walls are in rubble stone masonry on the ground floor. The original overhanging façade on the upper floors is a three-dimensional structure with braced frame timbers with Saint Andrew's crosses, vertical post (prumos) which frame the opening and the upright diagonals (travessenhos). The timber vertical posts reach a height of more than one floor. The infill of the triangular voids is composed of bricks and irregular stones with lime mortar.

DIACRONIC VISUAL COMPARATIVE ANALYSIS
In this research, it is shown the importance of integrating multisourced data for a 3D remote reconstruction of historical buildings, which have suffered numerous changes, e.g., after intrusive rehabilitation works or as a result of emerging cultural paradigm of modern life. In the selected case studies, comparing the historic photographs and the current state ( Figure 2, Figure  4, Figure 5), we note that the number of floors, the external wall configuration and the roof structures remain unchanged over the centuries, whilst other load-bearing construction components were changed, i.e., the external timber frame wall of the upper floors was replaced by hollow bricks and cement structure in both case studies.
It should be underlined the importance of the mixed construction system of the main façades in the history of building construction in Portugal and in Europe. In fact, the combined use of masonry and a timber skeleton in the upper floors evolves into a set of more informed solutions during the post-1755 earthquake rebuilding in Lisbon (Rossa, 1998;Mascarenhas, 2004). While the post-earthquake reconstruction in Portugal is the subject of extensive research and interest for years, this proto-anti-seismic type is little-known so far.
In case study No.1, when comparing the archival images of the building at the turn of 20th century, its main distinctive features remain unchanged, i.e., the overall building configuration, the opening layout, and the surface finishing (high chromatic compatibility of the restored building with the original).
Significantly, one visible alteration in the external façade regards the ceramic tile, dating back to 1930. A polychrome piece representing the Blessed Sacrament with two cherubs and an iron element to support the lighting system were placed in the main façade.
In case study No.2, high intrusive rehabilitation works occurred in 1940s under the Salazar dictatorship. As regards the aesthetic form, these works epitomize the reconstruction of an imaginary vernacular architecture during the Estado Novo (New State). The comparison between historical photographs and the current state ( Figure 5) shows the loss of original architectural elements, such as non-compatible plaster in composition and colour, the removal of the timber boards covering the double overhangs, as well as the introduction of projecting oriel window with carved wood, following the Islamic tradition (muxarabi). Indeed, this architectural element was used in the Middle Ages in Lisbon, and it is still visible in other buildings (e.g., in Rua Benformoso no.101-103, Lisbon). Resulting from the low importance of preserving integrity and structural authenticity, the external timber-frame walls on the upper floors were replaced by a reinforced concrete structure, hollow bricks, and cement mortar in 1940s, as proven by consulting the building dossier in the municipal archive (Stellacci et al., 2016).
It should be pointed out in this framework that "Integrity is a measure of the wholeness and intactness of the natural and/or cultural heritage and its attributes. Examining the conditions of integrity, therefore requires assessing the extent to which the property: a) includes all elements necessary to express its outstanding universal value; b) is of adequate size to ensure the complete representation of the features and processes which convey the property's significance; c) suffers from adverse effects of development and/or neglect" (UNESCO 2005).

FEATURE MATCHING AND GEOMETRIC VALIDATION
For case study No.1, feature detection and extraction find sparse feature points in the image (Figure 8) (first step of the SfM pipeline). After a preliminary automatic selection of feature points, point of interest for the next phase of the metric evaluation are manually selected, following the workflow defined by Condorelli et al. (2019). In the second step, feature matching and geometric verification find correspondences between the feature points in all the images (Figure 9).  Figure 7).
As result of the matching, a point cloud is obtained (Figure 9). Although being sparse, it includes the points used for the metric evaluation. A known distance was used for scaling the model (shown in Table 1). Table 1 shows the comparison between the distances measured in the sparse point cloud, obtained from the previous step of the workflow (the SfM pipeline), and the same distances measured on the current building ( Figure 10), during an on-site survey. Common points are analysed to evaluate the accuracy of the 3D virtual building model.   Results show minor differences, which can be neglectable from metric reconstruction. However, the comparison was on the common point between pre-and post-the rehabilitations work. Figure 12 shows the rectified building façades of case study No.2, using an archival photograph (1889)(1890)(1891)(1892)(1893)(1894)(1895)(1896)(1897)(1898)(1899)(1900)(1901)(1902)(1903)(1904)(1905)(1906)(1907)(1908). The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France The metric evaluation was performed comparing the same measure distance of these rectified images and the design projects, using the municipal building dossier (AML, Dossier no. 21582) (Figure 12 and Figure 13). Given the few differences obtained from residuals shown in Table 2, the results obtained from this remote survey are reliable, even considering that the reference distances were taken from a building dossier with an acceptable degree of tolerances.

CONCLUSIONS AND FUTURE CHALLENGES
This research addresses the remote reconstruction of traditional buildings obtained or validated by using data from (textual and visual) archival documents and onsite building survey. The main aim of this study is to share and discuss an advanced research methodology and its scalability in far-reaching applications on architecture, building engineering, and history related fields, including critical heritage sites. Built heritage that are highly prone to change due to real estate pressure, natural hazards or armed conflicts can be virtually reconstructed to unveil changes and permanence over time.
Looking at the broader framework, we connect heritage data with emerging needs related to digital transformation, including new-learning devices and integrated heritage policies.
Although each (virtual) building reconstruction requires a specific and contextual method, the proposed workflow could be a benchmark for future studies. The relevance of this research relies on the fact that the 3D reconstruction allows displaying the original configuration and the building changes by combining multiple reliable sources, as shown in case study No.2. The construction and architectural features of this historical building was heavily changed. Unfortunately, its main features, as originally conceived, cannot be restored, yet this remote survey allows to reduce biases on traditional building structures, e.g., those related to the indiscriminate use of muxarabi or iron support of cantilevered structures. Visual contents are impactful and overcome language barriers. The application of an integrated approach of 3D reconstruction and remote survey using AI algorithms can contribute to guarantying a truly tourists' perception of authenticity in cultural heritage sites or, hopefully, more sensitive heritage management. Further research studies are necessary for validating the reliability and the scalability of this photogrammetric methodology and for implementing its integration into GIS maps or geotagged apps. These advanced and freely accessible application, if designed for broader communities, e.g., tourists/visitors, local communities, and newcomers to the city, can raise awareness about the preservation of heritage authenticity and integrity, collective and private memories in historic cityscapes.