LOW-COST 3D ACQUISITION OF GEOMETRIC DATA FOR LIVING HERITAGE: ATTEMPTING TO RECORD THE PUDHU MANDAPAM, MADURAI

The driving forces behind the rapid development of accessible 3d modelling acquisition are generally economic. As the requirements for on-site data acquisition technology become cheaper and more user friendly, opportunities for the geographic dislocation of expertise become more viable. In effect, much of the diagnosis of a monuments’ morphology or condition can be made remotely, as a virtual model is constructed. This potential portability serves to reduce the impact, invasiveness and cost of survey and documentation processes. In cases of contested heritage conservation practices, the simple act of photographic recording can cause concern. However, photogrammetric recording is eminently advantageous for its capacity to provide non-destructive means to consider degradation and condition mapping as well as to record and monitor change over time. Here, two rapid surveys taken with portable 360° cameras a year apart, demonstrate the potential value and limitations of deploying recent techniques in order to deliver credible or useful survey data in a highly complex pillared hall that is intensively occupied.


INTRODUCTION
This paper relates a sequence of attempts made to develop a practical and workable methodology for the digital documentation of a living heritage site that is both geometrically complex and intensively occupied. It touches upon the emergent challenges and inherent controversies that relate to the role of digital documentation and the role of such data in the framing and interpretation of subsequent heritage governance. A comparison with the use of Terrestrial Laser Scanning in a contrasting managed heritage site of similar architectural quality and multicolumnar configuration is made. The merits of low cost 3d data acquisition, its implicit accessibility and its limitations and dependencies are presented and discussed. Addressing Living Heritage it touches upon issues beyond the scope of this paper, yet suggestive of meriting further discussion.
Using case studies of two geometrically complex historic interiors in Tamil Nadu with ritual roles, the relative merits of 3d acquisition methods and their representation to serve a range of challenges are examined. One is a relatively uninhabited managed heritage temple site and the other an intensively populated hall. Both are densely pillared and inherently challenging to survey using conventional laser scanning. In one, layers of physical adaptation and constant intensive human activity demand correspondingly inclusive methods of recording. In the other, the artefactual bones are more readily apprehended. Here, the virtues and utility of a range of current digital acquisition methods are evaluated in terms of their capacity to contribute to the documentation of tangible and intangible heritage, condition mapping and structural survey as well as visualisation for contemplation in support of non-destructive decision making. For the purpose of multilateral decision making and accessibility there are potentially further opportunities. The issue of lower cost also increases capacity for user generated workflows such as that developed by Rahaman et al (2019).
The examples presented here offer instances where both the methods of data acquisition and the provision of digital accessibility are challenging. However, each are representative of differing circumstances of heritage management and control. Firstly, in order to meet the challenges of data acquisition with limited access, it was necessary to deploy a range of tools which have been combined in different ways. For both sites, both Terrestrial Laser Scanning and traditional photogrammetric methods have been used.
At the Airavateshvarar temple, Darasuram, the conditions were relatively benign insofar as the granting of access for a drone LiDAR at was possible in conjunction with Terrestrial Laser Scanning of the interior. In addition, the built heritage is not intensively occupied there, making the recording simpler in terms of changing circumstances. That said, the physical built heritage there is apparently not as significantly at risk as that at the Pudhu Mandapam. The Airavateshvarar temple is under the management of the Archaeological Survey of India and has benefitted from a comprehensive repair programme. Its geometric completeness and lack of ambiguity as a model renders it a fine opportunity to support both geometric and diachronic analysis. Indeed, potentially there is scope here to develop Diachronic Semantic Models as posited by (Magnini et al 2019).
Ethical positions in Archaeology continue to seek revisions to object-centred approaches and to move towards people-centred paradigms for the inclusive interpretation and stewardship of heritage (Dru Evan, 2012). The shifting ground of authority in the context of attempts to decolonize the historicity of heritage is now overturning imperial monuments. The need to develop methods to support diachronic analysis (Daniel, 2020) which can acknowledge change and promote inclusivity is urgent. The place amongst this of advancing and increasingly accessible means for digital documentation, visualisation and participation is instinctively within reach. These requirements build on concepts of the aura as described by (Latour and Lowe 2019). However, there are both limitations and possibilities in the context of rapid developments.
Conventionally measured survey and architectural history would form the basis of the documentation of tangible heritage and ethnographies support that of the intangible. Increasing accessibility of digital methods of recording promises the potential for the integration of these. Here the limitations and potentials of this are examined from a series of attempts at two sites. The Pudhu mandapam or Vasanthamandapam which adjoins the vast Meenakshi-Sundareshvara Temple, Madurai, Tamil Nadu, India was built between 1628 and 1635. King Thirumalai Nayakar intended it to be a summer home for the Goddess Meenakshi and Lord Sundareshwarar. Its pillared hall is intricately furnished with carvings depicting scenes from sacred and puranic texts. Since the eighteenth century it has been occupied by tailors and shop stalls whose activities continue alongside its active ritual role in the Chitairai festival and as well as for urban gatherings. The degree to which the building operates in a commercial capacity for traders alongside its ritual role is evidenced on Google maps where the building is described as a "Shopping Mall". Images from 1858 and 2019 (Figures 2  and 3) illustrate the relative increase in inhabitation of the site.

CHALLENGES
An ideal plan for surveying the magnificent granite carved columns of the seventeenth century hall at the Pudhu Mandapam with a laser scanner would have been to clear the building of all its stalls, electrical installations, canopies, railings, and people. A network of scans would be plotted and cross referenced using a total station. In addition, in order to capture the crevices and concave forms that are not visible from below, the same regimen would need to be carried out as a series of levels, requiring a mobile platform or rig. This work would take in the region of 40 hours for 2 operatives on site and would require mobile lighting rigs, security and most challenging of all, unhindered access for a consecutive period of 5-6 days. In addition to this the external faces of the building would need to be cleared of all canopies, railings, market stalls vendors, tailors and people including pilgrims, tourists and the public. Alternatively, or additionally, a detailed photogrammetric survey tethered to such a geodetic network, deploying drones to capture occluded details, might create a rendered surface mesh more swiftly.
The resultant model could provide a geometrically accurate survey of the physical heritage asset enabling a certain degree of structural calculation using finite element mesh analysis, condition mapping and be the datum of a model to record future change or alteration. It would enable the detailed and thoughtful design and instruction for the layout of services and the location of any markers for monitoring. It would fail entirely however, to capture or record the living heritage of the site at all, or to inform future generations of the state of activity in that place in our time.
Even if granted unhindered access, the geometric complexity of the Pudhu Mandapam's 126 sculpted columns, their constant occlusions and high contrast between interior darkness and brightness outside would be a significant task to record with LiDAR. There are currently 235 shops operating between the columns of the pradakshina, the circumabulatory path around the hall. Adding to that, a central area that is inaccessible, its constantly fully occupied commercial and ritual activity and use as a public thoroughfare make the deployment of LiDAR unfeasible.
Figures 5 and 6 illustrate both the survival of the tangible heritage and the intensiveness of current use of the building as a "shopping mall". There is evidently a long tradition of criticising the way in which the building has been cared for. Interestingly, even in 1868, Lyon in his 'Notes to Accompany a Series of Photographs Prepared to Illustrate the Ancient Architecture of Southern India', described the carvings as having been "dreadfully injured by whitewash and painting of the vulgarest description which has been applied quite recently" (Lyon 1870). Caretakers of the Pudhu Mandapam today are justly wary of "experts" seeking to control or direct their practices. The role of digital documentation in this instance today takes on a much more complex task both technically and ethically of seeking to respect all these layers. It is precisely the co-existence of the historic and the continuing practice and activity which inspires marvel today. The quality and age of the historic photographs is extraordinary and to a great extent they provide the requisite documentation of the stone building.
Complex layers of governance and tenancy provide absolute limitations to expectations of access for experts to make surveys. However, there are credible risks in the form of nearby experience of destruction by fire that appear to threaten the resilience of the building. These events prompt concern to deliver the means to evaluate current conditions and practices reasonably. Thus collation of 3d models using hand-held 360 video footage became the tool of least intervention. Not simply for its non-destructive impact with respect to the tangible, but also to the intangible heritage of the site, including its capacity to record sounds. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China

A contrasting case; Scanning the Mukha Mandapa of the Airavateshvarar Temple, Darasuram
There are other similarly ornate buildings with pillared halls or mandapa in South India in which such challenges occur, where sites have been officially recognised and repaired and are no longer actively operating to such intensive degrees. For example, the Sri Ranganathaswamy Temple, also known as Thiruvaranga Tirupati has similar horse drawn pillars but as it has been a designated World Heritage site since 2014, there are no stall holders there. It could be argued that the existence of such physical twins might indeed be paralleled with the current notion of "digital twins". To some extent the UNESCO grouping "Great Living Chola Temples" is problematic insofar as the restoration interventions which heritagise the sites inevitably impact their "living" status. The complex history of temple restoration in Tamil Nadu is covered in (Branfoot 2013). The task of digital documentation in buildings that are not intensively occupied is inevitably simpler in practical terms. At the UNESCO World Heritage Site of the 12 th Century Dravidian Airavateshvarar Temple at Darasuram, Kumbakonam, it was possible to scan the interior of the recently restored 48 pillared hall, the mukha-mandapa with a terrestrial laser scanner (FARO X130 3D) at a high resolution. The greatest challenge being to ensure a sufficient number of corresponding scans were made between the columns. The output is a pointcloud which provides a static record enabling the extraction of orthogonal views such as a detailed reflected ceiling plan which hitherto would not have been possible to deliver.

Justification of alternative methods
It is therefore for reasons both of expediency and necessity that the fragmented nature of our survey forced us to elaborate on roughly hewn data and to reflect on its potential and its limitations. The use of 360° video images as a method for rapid assessment has some immediate opportunities with respect to recording certain features, for example, to count the number of merchants and their type, for instance. Indeed, this would be hard to achieve with a conventional apparatus by virtue of the fact that it can change daily and people are themselves mobile occlusions.
It is unlikely that a complete laser scanning survey would take place whilst the shops were open but no people were present, or that a sufficient number of scans could be taken in one day to achieve it. In this respect, the 360 method demonstrates an advantage over the laser scans. It also potentially enables the location of 2D images, both contemporary and historic, whose limited number may preclude the construction of 3D models but whose quality and level of detail are still of value. Indeed, the ability to locate precisely the selected point of view taken by historic photographers, offers new potential for analytical reflection on their perspective upon the same heritage asset. This is possible both literally in terms of considering the priority of their selected physical position with respect to a sculpted deity or ensemble but also in terms of their symbolic positioning with respect to creating formal readings of the iconographic hierarchy of the item or place.

Figure 8. Flowchart and timeline of 3d data acquisition
The flow chart (Figure 8) indicates how the initial three laser scans (Two at the east end and one at the west end, made in 2018 were subsequently used to scale the pointclouds from all traditional photogrammetry and spherical photogrammetry.

Laser scanning and preliminary data capture
In 2018, a total of 14 scans were made of the Temple complex with just three laser scans of the Pudhu Mandapam using a ©FARO X130 3D scanner, two at the East end and one externally at the West end (Fig 1). It was possible to register these using ©FARO Scene software and then to use these to scale all subsequent tasks. Although the scans are at a significant distance from one another and the alignment is not optimal, their role as a geometric control for the subsequent photogrammetry is critical.  The subsequent registration and alignment of these images was carried out using ©Agisoft Metashape Pro. Detailed elements were successfully identified, where the view was unobstructed, however the overlaying of multiple structures (see Figure 10) and objects made the creation of a mesh an overwhelming challenge. Figure 11. Pointcloud of the West front of the Pudhu Mandapam derived from traditional photogrammetry using ©ContextCapture Meanwhile at the East end, where 2 detailed laser scans were available 976 images were taken using a Nikon D610. The pointcloud of the East end was successfully exported into a mesh using BentleyContextcapture. However, creating a mesh of the West end proved more challenging owing to the number of occlusions and the sheer complexity of the physical geometry of overlayed structures which were often re-arranged.  The North and South Aisles were systematically photographed at approximately 1m strides looking up, ahead and down heading both East and West and also recorded using 4k 2D video. Again, however, the lack of capacity to capture images at high level had a significant impact on the resultant quality of the pointcloud that was created. Indeed the sheer magnitude of the task was vastly increased by the visual complexity of the shops and stalls themselves and the contrast in the illumination of products on sale as opposed to the unlit column capitals above.
However, seeking to test the viability of the relatively low-cost and swift acquisition of 360 image capture further, our key challenge was to try to create a pointcloud using data derived from these images themselves.
The question of whether the task was to record the historic tangible heritage or the living tangible heritage was impregnable. Indeed, it became evident that beyond the basic geometric scaling afforded by the laser scans, the visual seamlessness of 360 imagery was in fact both more accurate and more achievable.
In 2018 a 360 video was made using a Ricoh Theta V, this was useful in simply enabling an appraisal of the number and identification of columns and stalls. In 2019, returning to attempt a more thorough photogrammetric survey, these 360 videos were repeated with the same camera and subsequently converted into 360 video tours using 3d Vista. The value of these as a means to encapsulate not only the layout and activity but also the sound and as a means to integrate other data is significant, particularly as 360 video navigation becomes increasingly accessible.

3D MODELLING USING 360 IMAGES
360° cameras provide images and videos based on the equirectangular projection, which is also called latitudelongitude or spherical projection. The camera used in this work is a Xiaomi Mi Sphere. It captures 3.5K HD videos and 24megapixel photos using two front-and rear-facing wide angle sensors of 1/2.3 inches. Images are then stitched using the PC software Mijia Mi Sphere 360.
The advantage of 360° cameras is the opportunity to point the camera to any direction, and the entire space around the sensor will be captured. On-site data acquisition was carried out using a set of videos instead of taking multiple images. Although static images feature a better resolution, the complexity of the site would require a huge number of images, which does not seem feasible for the conditions inside the building. The acquisition of different videos is simply carried out walking inside the Pudhu Mandapam and holding the camera with a selfie stick, so that it stays higher than people, reducing occlusions and capturing more details of the structure.
4.1.1 Use of 360 Videos: 6 HR videos were acquired in about 10 minutes. Photogrammetric data processing was carried out extracting single frames at a fixed rate, obtaining 1532 equirectangular projection with a resolution of 3840x1920 pixels. Figure 15 shows three frames. The distance between consecutive frames is very short (about 0.3 m), but this is a fundamental requirement to allow the acquisition of images with sufficient overlap in small and narrow spaces, such as the corridors of the Pudhu Mandapam. This is another confirmation that a videos and the automatic extraction at a fixed framerate is a suitable solution for the documentation of long and narrow spaces.

Image orientation and alignment
Image orientation was carried out with ©Agisoft Metashape 1.7, using the cloud-based processing service Agisoft Cloud. After the creation of the project on the desktop application, the project can be uploaded on the cloud service. When the processing is finished, the result is automatically downloaded. Figure Y shows a plan of the computed camera positions and the extracted tie points coloured using the multiplicity, i.e., the number of images in which the same point is visible. As can be seen, the value is always larger than 9 and this confirms the ability to match the same point on multiple frames for the wide field of view of 360° images. The extraction and matching phase of tie points took more than 22 hours, and bundle adjustment 18 minutes.
The project was scaled using a known distance retrieved from the available laser scans at the east end of the building. The RMS of the project was 0.44 pixels, and all 1532 frames were oriented. The figure also shows some views of the camera location in 3D

Results:
An interesting result is also related to the calibration parameters of the camera. In the case of spherical images, the only calibration parameter considered is the focal length, which can be computed before processing as the width of the image divided by 2π. Indeed, the focal length corresponds to the radius of the spherical image (i.e., 611.15 pixels). ©Metashape does not apply additional corrections for image distortion. Figure P shows the image residual pattern (some red lines were cut in the proposed visualization). 3 areas with small residuals are clearly visible. Residuals are higher at zenith and nadir points (+/-90°), but also along the stitching lines of the two fisheye images. This is an overall indication that the spherical image obtained by mosaicking two fisheye images has a worse geometric quality in these areas when used for photogrammetric projects. The generation of a dense point cloud took about 2 hours. The point cloud generated with ©Metashape had more than 21 million points and was exported and manually cleaned in Autodesk Recap ( Figure 19).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China Figure 19. The cleaned point cloud in Autodesk Recap.

DISCUSSION AND FURTHER WORK
There is a crisis of heritage authority: technical, cultural and political in Tamil Nadu, where mass rapid urbanisation has risked swamping the physical dominance of its temple architecture, authority for conservation practices is contested between religious, political and local stakeholders. UNESCO have instigated a fact-finding mission after a report condemned over zealous conservation practices in 2017 (Times 2017). Byrne's definition of heritage as "social action" is relevant to the situation here (Byrne 2008). Further seeks to untangle complex justifications for replacement work (Mills 2020).
For the purpose of multilateral decision making and accessibility there are potentially further opportunities. The examples presented here offer instances where both the methods of data acquisition and the provision of digital accessibility are challenging. However each are representative of differing circumstances of heritage management and control. At the Airavateshvarar Temple, the ability to apply analysis to 'clean' digital 3d models for interpretation is greater but also by definition, limited. It should be acknowledged first that the role of the three initial laser scans in providing the capacity to geometrically scale the pointclouds was absolutely critical to the successful implementation of this task. The increasing affordability of 360 cameras and the accessibility of registration software appears bound to accelerate the use of these methods. The combination of navigable 360 videos for visualisation and interpretation with 3D pointclouds derived from the same swift acquisition process has patently desirable applications for the recording and modelling of complex heritage sites. The extreme limitations of access and profound geometric challenges here should indicate the greater potential for successful deployment in less challenging scenarios.
For living heritage there is increasing sensitivity to the risk of global appetites for the heritagisation of monuments overlooking and eclipsing the rights of local practices. The obvious injustice here brings urgency to the quest to develop non-destructive means to contemplate and negotiate conservation management strategies. The aim in ongoing work is to test the extent to which 3d models can deliver accessible and non-destructive tools for heritage management consensus building. The desire to use digital models to provide a platform for accessing other related data is well covered in the disciplines of interactive museums and also for the purposes of HBIM.