COMBINING 3 D VOLUME AND MESH MODELS FOR REPRESENTING COMPLICATED HERITAGE BUILDINGS

This study developed a simple but effective strategy to combine 3D volume and mesh models for representing complicated heritage buildings and structures. The idea is to seamlessly integrate 3D parametric or polyhedral models and mesh-based digital surfaces to generate a hybrid 3D model that can take advantages of both modeling methods. The proposed hybrid model generation framework is separated into three phases. Firstly, after acquiring or generating 3D point clouds of the target, these 3D points are partitioned into different groups. Secondly, a parametric or polyhedral model of each group is generated based on plane and surface fitting algorithms to represent the basic structure of that region. A “bare-bones” model of the target can subsequently be constructed by connecting all 3D volume element models. In the third phase, the constructed bare-bones model is used as a mask to remove points enclosed by the bare-bones model from the original point clouds. The remaining points are then connected to form 3D surface mesh patches. The boundary points of each surface patch are identified and these boundary points are projected onto the surfaces of the bare-bones model. Finally, new meshes are created to connect the projected points and original mesh boundaries to integrate the mesh surfaces with the 3D volume model. The proposed method was applied to an open-source point cloud data set and point clouds of a local historical structure. Preliminary results indicated that the reconstructed hybrid models using the proposed method can retain both fundamental 3D volume characteristics and accurate geometric appearance with fine details. The reconstructed hybrid models can also be used to represent targets in different levels of detail according to user and system requirements in different applications.


INTRODUCTION
Detailed and highly accurate three-dimensional (3D) modeling, reconstruction, and visualization of historical monuments and cultural buildings or objects is one of the most commonly employed approaches in digital documentation and preservation of cultural heritage and their applications.Although digital 3D reconstruction or modeling of cultural heritage targets has been carried out for decades and a variety of related instruments, algorithms and softwares have been proposed and developed, it still remains as one of the most challenging topics in digital documentation of cultural heritage in terms of research and practice (Koller et al., 2009;Münster et al., 2015).In recent years, 3D point clouds have become one of the most popular data for describing complicated heritage buildings or structures in great details.These point clouds can be acquired directly with different sensors and instruments such as laser scanning (or LiDAR, Light Detection and Ranging) equipments, or generated from images captured using digital optical cameras and range (depth) cameras or sensors with close-range photogrammetry algorithms.
Digital documentation of cultural heritage usually requires highdensity point cloud data.In this regard, terrestrial laser scanning (TLS) has been a popular instrument of choice for data acquisition.However, LiDAR-based point cloud data acquisition systems are usually (relatively) expensive and time consuming.Recently, image-based point cloud generation with photogrammetric matching algorithms have emerged as a popular alternative for ⇤ {corresponding author} obtaining high quality 3D point cloud data (e.g., Alshawabkeh et al., 2010;Dellepiane et al., 2013;Koutsoudis et al., 2014) or used in conjunction with laser scanned data sets (Jamhawi et al., 2016).
For most applications, the point clouds are usually further processed to produce 3D models of the targets.A fast and economic modeling approach is connecting all points to form a network (mesh) describing the surface geometry, shape, and other property of the target surface.Another approach is to create a parametric, polyhedral, prismatic model deduced from the point clouds.
In order to take advantages of both approaches, this research developed a simple coarse-to-fine modeling framework to combine 3D volume and mesh models for representing complicated heritage buildings and structures.The developed model generation framework first reconstruct bare-bones models from 3D point clouds of complicated cultural heritage buildings.The bare-bones model consists of parametric or polyhedral elements of the target, which are used as the skeleton to describe the basic structure of a complicated building.Then mesh-based surface model patches are generated and integrated with the bare-bones model to describe complicated surface decorations or other non-skeleton objects.Using this strategy, the reconstructed hybrid building model will maintain fundamental structural characteristics as well as fine details of complicated surface objects.In addition, the model will also have multiple levels of detail for more efficient visualization and analysis according to different user and application requirements.

POINT CLOUDS AND MODELS
As described previously, 3D point clouds can be acquired directly using laser scanners or generated from multi-view images using photogrammetric algorithms.In recent years, structure from motion (SfM) and related algorithms (e.g., Faugeras and Lustman, 1988;Rothermel et al., 2012;Wu., 2011Wu., , 2013) ) have become an popular method of choice to generate 3D point clouds from multiple 2D images.In addition to freeware, commercial software such as Photoscan, ContextCapture (Acute3D) and Pix4D are also widely used to create 3D digital surface models of various sites and objects from ground-based or UAV (Unmanned Aerial Vehicles) images.This type of 3D modeling or reconstruction has also been successfully applied to cultural heritage applications (Dellepiane et al., 2013;Stathopoulou et al., 2015;Sun and Cao, 2015;Wenzel et al., 2013) for it does not require expensive equipments and has lower requirements on operations.
Whether the data are acquired using laser scanners or generated with image-based photogrammetric algorithms, and no matter how dense the data are, 3D point clouds are still discrete points.
Although the points may have accurate 3D coordinates, there is no topographic relationship recorded in the original data sets.Therefore, the point clouds are subsequently processed and analyzed to produce 3D volumetric or surface models.Recently, after acquiring or generating the 3D point clouds of a cultural heritage site, building, or object, a subsequent processing is usually employed in order to create 3D models of the target.A conventional approach is to construct CAD (computer-aided design) style models in parametric or polyhedral formats.Another emerging strategy is to directly connect the 3D points to generate triangulated or grid-based mesh networks as a digital surface model (DSM) of the target.Both approaches have their advantages and disadvantages as summarized and listed in Table-1.For example, on the one hand, parametric or polyhedral models usually requires less storage space and can be easily constructed as object or element-based models to maintain the structural characteristics of the targets but they are difficult to show complex surfaces or small structures well.On the other hand, mesh-based models can describe complex surfaces in fine details, but they usually have larger data volumes to store all vertices and connections.In addition, although mesh-based models can describe fine details of complex surfaces, they are difficult to maintain the elements or structural property of the targets.In order to take advantages of both approaches, this study developed a framework to integrate 3D volume (parametric or polyhedral) and mesh models for representing complicated cultural heritage buildings in multiple levels of detail.

PROPOSED METHODOLOGY
The proposed 3D modeling framework is separated into three phases.The first stage is the processing of 3D point clouds.One of the most important task in this stage is the segmentation of point clouds.Effective point clouds segmentation algorithms are available for different data sources, applications and other considerations (Grilli et al., 2017).The objective of segmentation is to partition the data into different groups pertaining to different structure elements, which will correspond to different model components of the building.
The second phase is to generate a bare-bones volume model consisting of parametric, prismatic, or polyhedral components.The process involves identifying required parameters to shape segmented point clouds into parametric or polyhedral model components describing individual building elements (Teo and Chiu, 2015) , and then connecting all components using CSG (Constructive Solid Geometry) or similar algorithms to form a barebones model representing the skeleton structure of the target.If necessary, additional levels of detail of the bare-bones model can be achieved using procedural (Koehl and Roussel, 2015) or projected generalization (Tsai et al., 2012) algorithms.Figure 1   The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W5, 2017 26th International CIPA Symposium 2017, 28 August-01 September 2017, Ottawa, Canada

EXPERIMENTAL RESULTS
The proposed strategy for generating hybrid models of complicated buildings was first applied to an opensource 3D point cloud data set to test its validity and performance.The open-source 3D point cloud data set was generated by Seitz et al. (2006) for comparing and evaluating different milt-view stereo reconstruction algorithms and is available at <http://vision.middlebury.edu/mview/data/>.The point cloud was generated from 312 images sampled on a hemisphere of a plaster reproduction of "Temple of the Dioskouroi" in Agrigento, Sicily, Italy and thus is noted as the "temple test case" thereinafter.

CONCLUSION
This study developed a coarse-to-fine strategy that combines parametric or polyhedral type models and mesh objects for representing complicated cultural heritage buildings.The developed modeling framework is separated into three phases, including segmentation, bare-bones skeleton model generation, and integration of mesh objects.Although still a work in progress, example results of two test data sets demonstrated in this paper indicate that  using the proposed hybrid modeling approach can significantly reduce the necessary data points to retain the fundamental volume and structural attributes and to preserve detailed elements and surface appearances of complicated heritage buildings.The proposed approach also gives the reconstructed 3D models multiple levels of detail to fulfill different user and application requirements.However, a reasonable and systematic LOD definition criteria should be established for this process to be carried out more smoothly.Future improvement of the proposed hybrid modeling framework will also include increase the robustness of mesh and parametric surface blending and increase the degree of automation.
is an example bare-bones volume model generated from a complicated point clouds of a Chinese style Land God (Tu Di Gong) shrine commonly seen in Chinese descendent communities across China and Southeast Asia.The bare-bones model not only represents the basic geometric shape and formation of the building; it also maintains the fundamental structural attribute of the building.

Figure 1 .
Figure 1.Point clouds of a Chinese style Land God shrine and its reconstructed bare-bones volume model.
Figure-2 shows an image of the plaster model and the preliminarily segmented point clouds.Figure-3 shows two views of the reconstructed bare-bones model.As displayed in the figure, the reconstructed bare-bones model has effectively represent the skeleton of the building, which also provides an intuitive presentation of the components making up the entire buildings.As a result, in addition to the skeleton shape and geometric characteristics, the model also preserves the fundamental structural property of the building.

Figure 2 .
Figure 2. Temple of the Dioskouroi and preliminarily segmented 3D point clouds.

Figure 3 .
Figure 3. Reconstructed bare-bones model from the temple 3D point clouds.The original points are also shown in the figure.

Figure- 4
Figure-4 displays the reconstructed full mesh model and a hybrid model of the temple test case.The full-mesh model utilized all original point clouds and consists of hundreds of thousand triangles.The hybrid model can significantly reduce the data amount

Figure 4 .Figure 5 .
Figure 4. Full-mesh and hybrid models of the temple test case.
irregular mesh surface model (c) regular mesh surface model 6.Point clouds of the North Gate test case and reconstructed mesh-based surface models.

Figure 7 .
Figure 7. Polyhedral (left) and hybrid (right) models of the Taipei North Gate providing different levels of detail.