A VIRTUAL RESTORATION FRAMEWORK FOR THE” MOON GATE” BASED ON A MODIFIED SCALE OF EVIDENCE

The virtual restoration was one of the important protection approaches for the Great Wall. With the help of this type of technology, the damaged Great Wall can be restored to the previous state of the Great Wall at a specific time node so that the tourists can visit the Great Wall before the damaged in the computer. This this paper presented a virtual restoration framework for the” Moon Gate” located on the XinGuangWu Great Wall so that virtual recovered results can guide the researcher to repair the Great Wall in the real world without the secondary damage. This method can be divided into 4 parts: (1) collection of evidence based on the modified scale of evidence; (2) the fusion of DIM point clouds; (3) excavation of the reconstruction technology from the collected evidence; (4) the generation of 3D model. The experimental results showed that the maximum error of the reconstructed “Moon Gate” was 1.088cm without considering the brick seam and our proposed virtual restoration framework of the Great Wall was robust and accuracy, and it has certain reference significance for guiding the restoration of the damaged Great Wall. * Corresponding author


INTRODUCTION
The Great Wall was one of the world's most impressive Cultural Heritages and was well known as one of "Seven wonders of the world". In 1987, it was inscribed to the United Nations Educational, Scientific and Cultural Organization (UNESCO) World Heritage List. Suffering from the natural disasters and man-made destruction, the protection for the Great Wall was imminent (Ming Ming, 2012;Xuefeng Yuan, 2019). The virtual restoration was one of the most important protection approaches for the cultural heritage. With the help of this type of technology, the damaged cultural heritage can be restored to the previous state of the cultural heritage at a specific time node so that the tourists can visit the cultural heritage before the damaged in the computer. Moreover, the virtual recovered results can guide the researcher to repair the cultural heritage in the real world without the secondary damage. Nowadays, the virtual restoration have been used in the protection of various types of the cultural heritage and achieved a good performance (Adam Blitz, 2019;Pedro Martí n Lerones, 2014). Although the virtual restoration have some advantages, there were still some limitations in the process of the virtual restoration. One hand, the current virtual restoration lacked systematic and scientific classification of virtual restoration evidence which were indispensable for the virtual restoration, which makes it difficult to organize the evidence during the virtual repair process. On the other hand, the researchers mainly focused on recovering the appearance of the cultural relics. The level of real productivity and the construction process at the specific period of building the ancient site were usually ignored. This leads that virtual restoration results was not reliable. To solve these problems above, we take the collapsed "Moon Gate" of the Great Wall located in SuoZhou, ShanXi Province, China as an example and proposed a novel framework for the virtual restoration of the Great Wall based on a modified scale of evidence. In this framework, we modified the scale of evidence created by Tayfun Oner for the Byzantium 1200 project team(Rafael Ortiz-Cordero, 2018) as two types of evidence: direct evidence and the circumstance evidence depending on whether the evidence can be used for the virtual restoration separately and directly. In the stage of the collection of the evidence, the survey data and were collected as the direct evidence and the images collected from the all types of websites as the indirect evidence. Subsequently, these two types of evidence were used to recover the collapsed great wall. The flowchart of this paper was shown in Figure 1.

The Collection of Evidence
XinGuangWu Great Wall stretches for 10 km along rolling mountains, starting from XinGuangWu Village ShanYin Xian and ended in BaiWeiKou Village DaiXian. Moon Gate was generated from the 10th watchtower which have collapsed. It located on the main mountain ridge. From the foot of the mountain to the moon gate, the height was up to 800 meters. Figure 2 showed the appearance of "Moon Gate" before and after collapsed.
(a) (b) Figure 2. The moon Gate (a) befor collapse;(b) after collapse The west and south sides of the 10th platform were steep, and the traditional methods of surveying and mapping were difficult to collect the existing complete sites. In this paper, the data were collected from an airborne camera imaging system (PHANTOM 4 RTK). This system was composed of a FC6310R camera with a 13.2*8.8 mm 2 sensor size and a 2.41μm pixel size, which can capture 20 million pixels. This system provided the photogrammetry five-way flight function. In one job, it acquires a type of images (with 8.8-mm focal length) that point in a direction. The flying height above the watchtower was approximately 85 m. Thus, the ground sampling distance (GSD) for all cameras ranges from 2.5 cm. The baseline between two consecutive images was approximately 40 m. The distance between two neighboring stripes was approximately 60 m, which provides highly redundant ground feature observations during aerial triangulation and point cloud generation. Moreover, 4 points (3 points were regarded as the control points and 1 point was regarded as the verification point) were collected by GPS RTK for SFM. To obtain the circumstance evidence of the virtual restoration for the moon gate, the network crawler technology was applied to obtain the image data of the XinGuangWu Great Wall before collapse from tourism, social, local information portals and other related websites as was shown in Fig.4. In this process, the key words "ShanXi Ming Great Wall", "XinGuangWu Great Wall", "Moon Gate", "ShanXi Moon Gate", "Ancient Great Wall", "Great Wall Site", "ShanXi Ancient Great Wall", "Ming Great Wall Moon Gate" and "Collapsed Great Wall" were used. Notably, the time when the images were generated ranged from 2014 to 2016 so that the shape of "Moon Gate" can keep stability. After the collection of the photo, the manual ways were still needed to wipe off the false images, and then further obtain images with camera parameters. The collected images was shown in Figure 5.

The Generation and Fusion of DIM Point Clouds
Due that the shape of "Moon Gate" has changed before and after the collapse and the scale and the viewpoint where captured the images between the searched images from the website and the images collected by UAVs, the aerial triangulation of all the collected images was very difficult. Therefore, we applied the dense matching technology to generate dense image matching (dense image matching, DIM) point clouds from two types of images separately, and then the two types of DIM point clouds were fused using the point cloud matching algorithm with the help of the overlapped regions.
In this paper, we applied the commercial package Agisoft PhotoScan (Dong Y.Q., 2018) which was able to automatically orient and match large datasets of images to generate the DIM point clouds. Due to commercial considerations, little information was available concerning the internal algorithms employed. Hence, we described the process of the generation of the DIM point clouds based on the workflow of Agisoft PhotoScan. Specially,  Add the original images, the positioning and orientation system (POS) data and the camera calibration parameters into this software.
 Align photos. In this step, the point features were detected and matched; and the accuracy camera locations of each aerial images were estimated.
 Build the dense points clouds. According to a previous report, a stereo semi-global matching like (SGM-like) method was used to generate the DIM point clouds. After the generation of the DIM point clouds, we fused these two types of DIM points from the images collect by UAVs and images selected from the website so that we can get the geometric information of "Moon Gate" before collapse. Before registration, select 4 common points (as shown in Figure 6(a)) to rough the two point clouds for a better initial parameter. Although the scale has been adjusted in the data preprocessing, the difference in the scale between the two points still exists. To obtain better matching performance, we applied the Scaling Iterative Closest Point (SICP) algorithm (Chen Elvis C. S., 2015) to match the point clouds P to the target point cloud Q. This algorithm can be expressed as equation (1) In this formula, S=diag(s1,s2,…,si,…,sm), was a non-zero scale matrix, R was an orthogonal matrix, and t was a translation vector.
 , j=1,2,…,Nq. The implementation of this algorithm was an iterative process. The matching results can be seen in Figure 6(b).
(a) (b) Figure 6. (a) The selected common points for the initial registration; (b) the registration results using SICP algorithm

Excavation of the Reconstruction Technology from the Collected Evidence.
The reconstruction technology was the key of the virtual restoration. It not only can help us to understand the development of social productivity but also can provide the details of the reconstruction. We can make use of the collected evidence to obtain the reconstruction technology.  As was shown in Figure 7, the reconstruction technology mainly contained the materials and the structure of "Moon Gate" in this paper. From the texture of collected images, the "Moon Gate" was mainly built with external outside brick and inside earth. The earthen material was mainly composed of fine grain and gravel. The structure of "Moon Gate" was divided into two parts from the bottom to the top. The lower part was the practice of connecting the outsourced masonry wall with the inner earthen layer, and the upper part was the practice of building outsourced brick for the earthen plate. The reconstruction technology can be used for the generation of 3D model.

The Generation of 3D Model
From the analysis, The "Moon Gate" consist of 134 bricks and 27 earthen plates which can be seen directly before collapse. According to the construction technology, other bricks and earthen plates were calculated. Different elements were shown by different colours.
(a) (b) Figure 8. (a) The irregular earth plate; (b) the constructed model of moon gate before collapse The process of generating the 3D model consist of two steps. At first, we make use of the fused DIM point clouds to build an irregular triangular network by the commercial software geomagic studio as was shown in Figure 8(a). Subsequently, the built irregular triangular network was imported into the 3ds max software for three-dimensional modelling manually. The result can be seen in Figure 8(b).

EXPERIMENT
(1) The generation of DIM point clouds During aerial triangulation, the total number of GCPs measured was 3, and the number of verification points was 1. The root mean square (RMS) values for both the control and verification points were less than 2 cm in the X-and Y-directions and in the Z-direction. The projection errors (standard deviation of image coordinates) were between 0.07 and 0.477 pixels for all points, which was likely due to a large OAI tilt angle that introduces larger geometrical deformations as well as the no modeled lens distortion errors. This demonstrates that the PHANTOM 4 RTK system can produce highly redundant object measurements. The point cloud generated includes natural RGB color. The normal vector as well as the image's interior and exterior orientation parameters estimated by aerial triangulation will be used for visibility verification during the back projection stage. Fig. 9 shows the photogrammetric point cloud generated and rendered in a 3-D perspective with natural color. The average point cloud density of our data was more than 200 points/m 2 .
(a) (b) Figure 9. The generated DIM points of moon gate; (a) after collapse; (b) before collapse (2) The reconstructed "Moon Gate" From the recovered point cloud data, the north-south and eastwest span of the base of "Moon Gate" was 14.77m and 8.26m separately. The thickness of each earth layer was about 10cm and the size of the existing brick around was 38 x 20 x 8cm to 40 x 26 x 10cm. The top of "Moon Gate" was consist of square green brick and the size was 32cm x 32cm x 8cm. The rest were the broken bricks. Due that the shape and scale of these broken bricks were different, these broken bricks were difficult to be calculated systematically. To evalute the accuracy of the reconstructed "Moon Gate", 6 bricks from the common part of the recovered model and the pre-collapse model were select. As was shown in table 1, RMS was 0.916cm and the maximum difference was 1.088cm without considering the size of the brick seam. The recovered model show that this method has a certain reference to the virtual restoration of the Great Wall sites.

CONCLUSION
In this paper, the three-dimensional model of "moon gate" located at the 10th watchtower of Shanxi Ming Great Wall was recovered in the computer by virtual repair technology. In the entire process, not only the appearance of the "Moon Gate" but also the reconstruction technology were considering. However, there were still two limitations. One hand, we only make use of two types of evidence and didn't evaluate the quality of the collected evidence; on the other hand, the entire model of the 10th watchtower were not recovered completely. Moreover, in the future study, we will focus on the automatic virtual reconstruction of the Great Wall.