INTEGRATING HETEROGENEOUS DATASETS IN HBIM OF DECORATED SURFACES

: There are multiple conservation challenges related to decorated surfaces, the majority are intimately linked to its documentation. This paper draws on wall paintings as a representative of decorated surfaces, arguing the importance of considering its fourth-dimensionality—space and time—in its conservation and documentation. To that end, we propose the use of Building Information Model (BIM) as a platform to consolidate this approach together with various documentation techniques used for the conservation and management of wall paintings. This paper exemplifies this method with a case study of Myin-pya-gu Temple in Old Bagan (Myanmar); firstly, reviewing the different techniques used to document the temple and wall painting (photography, photogrammetry, laser scanning, reflectance transformation imaging (RTI); and secondly, discussing the data integration within a BIM environment. This position proposes a transition from a two-dimensional to a four-dimensional approach in wall painting conservation, potentially opening up possibilities of documentation, monitoring, simulation, or dissemination. Ultimately, the case study of Myin-pya-gu has the objective to introduce the use of HBIM as a platform for consolidating the documentation of decorated surfaces.


INTRODUCTION
The conservation challenges related to decorated surfaces are manifold, and mainly related to material heterogeneity, variation in size, and its intimate relation to the structure.Wall paintings are an essential component of decorated surfaces, highly significant because of their intimate connection to its support and its relation to the context.This paper draws on wall paintings as a representative of decorated surfaces, arguing that its conservation in-situ should not only be concerned with the treatment for their current condition stabilization, but it should also consider its preventive conservation, acknowledging its supporting structure and its greater surrounding context (Cather, 1999).Additionally, the incorporation of time is essential to understand its complex physical history and evolution (Cather, 2010).Accordingly, the four-dimensionality of wall paintingsspace and time-is considered fundamental to its conservation and documentation.
In this paper, we introduce the possibility of using Building Information Model (BIM) as a platform to consolidate this fourdimensionality approach together with various documentation techniques related to the conservation and management of wall paintings.BIM allows communication among multidisciplinary professionals-wall painting conservators, art historians, architects, engineers, and archaeologists-and bridges the gap between 2D, 3D, and 4D documentation.A BIM provides the possibility to manage, store, and coordinate information from different sources.It also allows interoperability with other software for energy simulation or structural analysis.This paper aims to stress the need for a consolidated methodology documenting the wall paintings within its context in a precise manner.It also discusses several approaches to data integration for wall paintings within a BIM environment using the case study of the Myin-pya-gu temple in Old Bagan, Myanmar (Figure 1).Section 2 of this paper examines wall painting conservation and its current approaches.Section 3 discusses BIM and the methods for integrating different data types.Section 4 presents the case study of Myin-pya-gu and the elaboration of its BIM.Finally, Section 5 introduces a theoretical framework with the possibilities to explore between BIM and wall paintings.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W15, 2019 27th CIPA International Symposium "Documenting the past for a better future", 1-5 September 2019, Ávila, Spain

Understanding Wall Paintings
Wall paintings are defined as any painting for which the support is the structure itself (Cather, 2010).Components of wall paintings can generally be categorized according to the different functions of each layer.If necessary, on top of the supporting structure lies the non-structural plaster layer(s) to level and smooth the surface, and, in the case of fresco, to bind the paint layer (Brigstocke, 2001).On top of the plaster layers, the wall painting may include a sealant, ground, paint layers, and other superficial decorative metal foils and attachments.

Wall Painting Conservation Approach
For wall paintings, the two-dimensional (2D) painted surface extends into the third dimension (3D) of depth, that is the stratigraphy (Cather, 2010).The stratigraphic complexity is illustrated in Figure 2, which shows all the components of the wall paintings in Myin-pya-gu.As an integral part of the immovable cultural heritage, wall paintings are inseparable from the context that informs conservation decisions (Lowinger et al., 2010).Considering the physical history of wall paintings-the fourth dimension (4D) of time-the approach of conservation in-situ leads to a fundamental question of the critical assessment of the problems and their change; this assessment involves the assembly and interpretation of the condition over a period of monitoring time (Cather, 1999).In terms of interventions, 'Conservation is about allowing future options' (Lowinger et al., 2010).The importance of documentation lies not only in the above process, but also during and after the treatment, being essential throughout the whole process of conservation (Martin de Fonjaudran et al., 2011).

Wall Painting Documentation Approach
Advances in technology offer us a bewildering choice of equipment and software for documentation and investigation over the past few decades (Cather and Schimd, 2000).The documentation of wall paintings can be carried out using a wide range of different techniques, such as imaging with different light sources, automatic matching of scattered bi-dimensional datasets, and range-based techniques (Chiabrando et al., 2015).Amongst all techniques, photogrammetry is most widely used due to its low cost, speed, reliability, and non-invasive approach (Remondino and Campana, 2014).Range-based techniques such as Terrestrial Laser Scanning (TLS) represent advantages in the acquisition speed and the amount of data collected with both high accuracy and resolution.Other techniques for documentation and non-invasive in-situ investigation include portable microscopy, multispectral imaging, infrared thermography, and elemental/molecular analysis (Wong et al., 2012), which are not discussed in this paper.

Current Status and Limitations
Presently, the majority of information related to the conservation of wall paintings is obtained in the form of 2D drawings created manually or directly on electronic devices (Wong and Agnew, 2011).The challenge in integrating complex data in four dimensions is the fourth dimension of time.Accumulation of layers, alteration of appearance, and human interventions without proper recording can all add on to the complexity.Geographic Information Systems (GIS) have been used as a method to manage the conservation of a painting.
In Remondino et al., (2011) they combine the 3D information (laser scanning, photogrammetry) and conservation notes through 2D raster and vector formats in a GIS to monitor a painting over time.The multispectral photography and Digital Surface Models (DSMs) were introduced as layers in the GIS to allow the conservators to analyze changes in the painting (Remondino et al., 2011).

HISTORIC BUILDING INFORMATION MODELLING FOR WALL PAINTING CONSERVATION
The use of BIM focuses on the design and life-cycle management of new building, yet in recent years, its application has increased concerning historic buildings (HBIM) (Dore and Murphy, 2017).The general purpose of documentation is to supply accurate information that will facilitate conservation, monitoring, and maintenance for the survival of an artifact (Letellier, 2007;Eppich and Chabbi, 2007).BIM offers the possibility to 'incorporate both quantitative assets (intelligent objects, performance data) and qualitative assets (historic photographs, oral histories, music) and provide a navigable timeline that chronicles tangible and intangible changes in the past and projections into future' (Fai et al., 2011).Pauwels presents his approach of 'Architectural Information Modelling' to incorporate different sources of historical information, including photographs, scanned documents, and research materials (Pauwels et al., 2008).Simeone affirms that BIM 'provides a single database where the actors can add and edit data collected along the entire investigation process' (Simeone et al., 2014).One of the challenges of working with existing structures is the modeling of irregularities or complex geometries from the point cloud (Dore and Murphy, 2017).Two different ways have been explored to automatically or manually address this problem.Several publications propose different methodologies to address this existing gap with a high level of detail (LOD) and accuracy (LOA) (Banfi et al., 2017).
The advantage of a BIM approach-as opposed to a GIS approach-is that the 3D data can exist in its 3D dimensional format; it is not reduced to a raster or vector image.Moreover, for wall paintings, the structure of the building is an essential factor in its conservation.Thus, we propose HBIM as a platform for managing the documentation of wall paintings at a higher level of detail.The following subsections discuss the integration of different data types in a BIM environment.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W15, 2019 27th CIPA International Symposium "Documenting the past for a better future", 1-5 September 2019, Ávila, Spain

Laser Scanning
The integration of point clouds (Figure 3) from TLS in a BIM environment is seamless.Point clouds can be brought directly into most BIM software, and most modeling is done from this data.After elements are modeled, the point cloud can also be used to visualize differences between the idealized BIM, and the true geometric data (Macher et al., 2019).

Photogrammetry
Photogrammetric point clouds can be used directly for modeling in the same way that the TLS data is used.Photogrammetric point clouds tend to have better color representation when compared with TLS data, which is important when looking at painted surfaces (Figure 4).Mesh data produced in a photogrammetric pipeline can represent the small details of a painted surface better than BIM surfaces.However, the dataespecially if it is not decimated-is too complex to be integrated into a BIM environment.Alternatively, the meshes can be referenced to their corresponding BIM elements and viewed in an external software if necessary.

Photography
Photography is one of the most common methods to document decorated surfaces.Visual information-ranging from UV to VIS to IR-can tell conservators about not only the condition of a wall painting, but can give insights as to the physical and chemical composition of the materials as well (Verri, 2008;Verri, 2009).Images can be projected directly onto a surface in a BIM environment, but to do this with high-resolution images can make the model computationally heavy to use.A more performant approach is to reference the images to the model elements.For smaller details -for example, a crack in a wallthe photographs can be referenced to that specific location on the BIM element.

Reflectance Transformation Imaging (RTI)
RTI is a photography-based visualization technique that is used to understand low relief details on a surface (Figure 5).By taking a series of photographs from a still point, with varying light positions, a surface can be re-lit digitally in real-time.RTI is used to help conservators visualize the small details that are hard to see from a single photograph (Malzbender et al., 2000;Malzbender et al., 2001;Mudge et al., 2006).RTI requires a software for re-lighting, so, like the photographs, it has a reference to an element or location.Because photographs, photogrammetry, and RTI are all likely to be referenced within the BIM environment, it is useful to look at an external method to link all of these data types like the one in Pamart et al., 2019.Figure 5. Myin-pya-gu wall painting RTI data set.

Documents
Other forms of information (Figure 6)-drawings, analysis results, text files, readings, etc.-can be referenced to a specific location in the BIM in the same way as the photography (described in Section 3.3).Figure 6.Graphic documentation of an area in Myin-pya-gu.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W15, 2019 27th CIPA International Symposium "Documenting the past for a better future", 1-5 September 2019, Ávila, Spain

HBIM
Research HBIM is concerned with the integration of multiple datasets; usually from historic drawings/surveys, total station surveys, hand measurements, rectified photography, photogrammetry, and laser scanning measurements.Chow et al. (2017) assimilate heterogeneous datasets in BIM while developing a modeling verification system.Their work is displayed by their digital model of Centre Block of Canadian Parliament, modeled from primary sources: laser scans and photogrammetry, and secondary sources: archival drawings, photographs, historical steel catalogs, and technical reports.Recently, Macher et al. ( 2019) developed a method of visualizing the distance between a point cloud and a BIM wall.This method can be applied to the BIM integration of photogrammetric and laser scan geometric data from decorated surfaces and could be expanded to display deviations between point clouds gathered at different epochs.However, at this point and time, deviation mapping and visualization are limited to a DSM approach similar to the GIS approach of Remondino et al., 2011.The geometric relationship between the segmentation of architectural volumes (BIM) and the wall paintings is achieved not only through the color point cloud but from other color information.
Color information-captured with photogrammetry, laser scanning, rectified photography, and condition assessments-can also be integrated into a BIM by producing an orthophoto and applying it as a texture to a building element.Each of these elements visually represent a dataset and can always be linked to the original documents.The BIM phasing tools allow for the addition of different epochs of documentation; multiple versions of a building, or an element, can be represented at stages of its lifetime.

HBIM of Myin-pya-gu's wall paintings
The application of HBIM in the field of wall painting conservation may include integration of different aspects related to the conservation approaches, the simulation of conservation conditions, public education and dissemination.As discussed above, HBIM has been demonstrated as a valid and resourceful tool to develop projects of heritage building preservation.Moreover, its application in wall painting conservation can provide enhanced controllability in its documentation and condition monitoring process in the future, which helps to initiate new fields of research.This paper develops a HBIM of a section of Myin-pya-gu temple with particular attention to its wall paintings.
The HBIM of the temple incorporates different datasets (point cloud, photogrammetry, photography, and RTI) gathered and processed during the summer of 2018 by a multidisciplinary team formed by members of Carleton University, Department of Archaeology and National Museum of Bagan, and Mandalay Technological University.There is no information about other documentation campaigns of this temple; for that reason and its particular heritage value and difficult access, we propose it as a cornerstone to exemplify this four-dimensional approach.
All the datasets are scaled and oriented to a common control network in order to allow the incorporation of other datasets over time.Apart from that, it is also proposed the use of natural points.The section of the west corridor is selected to exemplify the integration of different datasets (Figure 7).The photogrammetry is integrated as both a point cloud and high quality orthophoto; the point cloud is registered to the laser scan point cloud and the orthophoto is applied to HBIM geometry surface as a texture.The RTI viewer does not have direct integration with BIM; however the color and surface textural layers -seen in figure 5 -are exported as image formats, applied to the HBIM geometry, and aligned to the photogrammetric orthophoto.The RTI is also used to create a condition assessment of specific area of wall painting which can also be layered as a textural layer or a vector CAD format (Figure 8).(Pichard, 1992).
By setting 3D views in the drawing sheets of the HBIM, laser scan, photogrammetry, RTI, and photography perspectives are matched to the 1992 photography.If new data is layered in the future, these same views can be used to match perspectives for comparison.Figure 9 shows the comparison between the 1992 report and matched perspectives from 2018 laser scans, photogrammetry, and photography.

Application in Conservation Approaches
HBIM has the potential of integrating 4D information to facilitate the wall painting conservation approach.The objective is to combine, in the same document, different heterogeneous datasets such as historical photos, condition assessment, analytical report, or previous interventions.
The scheme of wall paintings (and painted sculptures) can be clearly shown with a comprehensive 3D model (Figure 10).
Zooming in, the original technology of painted schemes is presented along with architectural features.Apart from understanding the original technology, HBIM is also promising in condition assessment-such as the plaster losses present in Myin-pya-gu.Another opportunity is the comparison of rectified images and the monitoring of the same area over time.
Finally, having the possibility to record previous treatments is critical for a heavily-compromised site as it is with Myin-pyagu.In conclusion, the generation of a highly detailed HBIM provides a sound basis for the documentation, monitoring, and intervention of decorated surfaces.The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W15, 2019 27th CIPA International Symposium "Documenting the past for a better future", 1-5 September 2019, Ávila, Spain

Simulation of conservation conditions
Because wall paintings are inseparable from their context, the ethical preference for preventive intervention in the conservation field requires a better understanding of how things are made and how they behave in open physical systems (Cather, 2007).Under the commonly limited conditions, it is difficult to conduct experiments for conservation approaches with controlled parameters.Therefore, some of the simulations that can be performed from the generation of the wall painting BIM are proposed below.§ Lighting condition: the alteration of photo-sensitive pigments and/or organic colorants on the wall paintings are related to the exposure to direct or indirect sunlight.§ Environmental condition: For passive and preventive conservation purposes, the microclimate of the wall paintings is often monitored, controlled, then tested in an iterative approach for stabilization.§ Visitation impact: Risks of tourism include not only physical damage, but more importantly the atmospheric changes.Visitor impact study undertaken at the Tomb of Nefertari is a pioneering study with abundant resources (Maekawa and Preusser, 1993).

Public education and dissemination
Producing a BIM can also aid the generation of animations, dynamic physical and virtual exhibitions for education and dissemination of cultural heritage.In the field of wall painting conservation, the multidisciplinary complexity and the consequent obligation of communication are fundamental (Cather, 1997).This leads to the necessity of public dissemination.There are numerous examples of digital documentation and BIM data being leveraged for public dissemination to increase public engagement in heritage places (Factum Arte, 2012;Barazzetti et al., 2015;Agudo et al., 2015;Fernández-Palacios et al., 2017;Kersten et al., 2017;Dhanda et al., 2017;Graham et al., 2018;García et al., 2018).

CONCLUSIONS
The conservation of decorated surfaces, especially wall paintings emphasizes the importance of documentation.This paper aims to improve the current procedures developed in wall painting conservation with the use of building information technology and the latest documentation techniques.As was stated in the introduction, the development from a twodimensional to a four-dimensional perspective is key to in-situ conservation within the context.The use of HBIM offers the possibility of an integration of all the information gathered from multidisciplinary professionals, as well as allowing interoperability.The HBIM can be potentially used to adequately improve the documentation of deterioration phenomena, original technology, interventions, and condition monitoring.Additionally, it opens up other possibilities such as the simulation of conservation condition.The case study of Myin-pya-gu has the objective to introduce the use of HBIM as a platform for consolidating the documentation of decorated surfaces.

Figure 7 .
Figure 7. Layered datasets of Myin-pya-gu temple.Within the HBIM, data sets are integrated from photogrammetry, laser scanning and RTI.The HBIM geometry is created by tracing the (colorless) laser scan point cloud.