RECORDING APPROACH OF HERITAGE SITES BASED ON MERGING POINT CLOUDS FROM HIGH RESOLUTION PHOTOGRAMMETRY AND TERRESTRIAL LASER SCANNING

Different approaches and tools are required in Cultural Heritage Documentation to deal with the complexity of monuments and sites. The documentation process has strongly changed in the last few years, always driven by technology. Accurate documentation is closely relied to advances of technology (imaging sensors, high speed scanning, automation in recording and processing data) for the purposes of conservation works, management, appraisal, assessment of the structural condition, archiving, publication and research (Patias et al., 2008). We want to focus in this paper on the recording aspects of cultural heritage documentation, especially the generation of geometric and photorealistic 3D models for accurate reconstruction and visualization purposes. The selected approaches are based on the combination of photogrammetric dense matching and Terrestrial Laser Scanning (TLS) techniques. Both techniques have pros and cons and recent advances have changed the way of the recording approach. The choice of the best workflow relies on the site configuration, the performances of the sensors, and criteria as geometry, accuracy, resolution, georeferencing, texture, and of course processing time. TLS techniques (time of flight or phase shift systems) are widely used for recording large and complex objects and sites. Point cloud generation from images by dense stereo or multi-view matching can be used as an alternative or as a complementary method to TLS. Compared to TLS, the photogrammetric solution is a low cost one, as the acquisition system is limited to a high-performance digital camera and a few accessories only. Indeed, the stereo or multi-view matching process offers a cheap, flexible and accurate solution to get 3D point clouds. Moreover, the captured images might also be used for models texturing. Several software packages are available, whether web-based, open source or commercial. The main advantage of this photogrammetric or computer vision based technology is to get at the same time a point cloud (the resolution depends on the size of the pixel on the object), and therefore an accurate meshed object with its texture. After matching and processing steps, we can use the resulting data in much the same way as a TLS point cloud, but in addition with radiometric information for textures. The discussion in this paper reviews recording and important processing steps as geo-referencing and data merging, the essential assessment of the results, and examples of deliverables from projects of the Photogrammetry and Geomatics Group (INSA Strasbourg, France). * Corresponding author.


INTRODUCTION
The topic of digitization Cultural Heritage sites is often related to interdisciplinary research projects.Such projects are linked to the operability of recording approaches by taking into account the user's needs in different fields as archaeology, architecture, geography, geology, etc.The approach is usually based on 3 steps: spatial data acquisition, modelling and texturing, management of spatial data.Currently, the two most appropriate recording techniques are photogrammetry and laser scanning, both able to deliver dense point clouds.In section 2, experiences based on the combination of photogrammetric dense matching and Terrestrial Laser Scanning (TLS) in Cultural Heritage are shortly reviewed.Section 3 is a summary of important methodological steps in both methods.Results from a few examples of recording are then discussed in section 4 and 5: a cave from the Bronze Age, a Roman Theatre, and a medieval castle.Finally, we draw the conclusion on accurate documentation by merging high resolution photogrammetry and laser scanning from these experiences.

RECORDING IN CULTURAL HERITAGE IS TECHNOLOGY DRIVEN
Actors working in the field of heritage documentation agree that recording, documentation, and information management are among all the central activities of the decision-making process for heritage conservation management and a fully integrated part of research, investigation and treatment (Letellier et al., 2007).The choice of the appropriate methods depends on the background and level of training or education of these actors in the field of surveying.TLS technology is more recent and changed the documentation approach from vector based models to point cloud based approaches (Grussenmeyer and Hanke, 2010).For a long time, laser scanning was the main solution to produce dense 3D point-clouds allowing high resolution geometric models, while photogrammetry was more suited to produce high resolution 3D textured models of small objects.Today, both photogrammetry and laser scanning can produce dense and accurate point clouds.
A review of papers related to the topic of merging TLS and photogrammetry data has been presented in Grussenmeyer et al. (2011).The choice of the most effective technique is not always obvious (El-Hakim et al., 2004;Grussenmeyer et al., 2008;Remondino and Rizzi, 2010).).There is no single method or technique capable of giving satisfactory results in all measuring conditions (Voltolini et al., 2007).It is therefore interesting to consider different levels of integrated laser scanning and photogrammetric data.Medieval castles are often used as test bench for modelling issues related to the automation of photogrammetric methods and to the fusion of 3D models acquired with different techniques, at different point densities and measurement accuracies (Voltolini et al., 2007;Grussenmeyer et al., 2008).Interesting documentation requirements and specifications about the actual 3D surveying and modelling techniques and methodologies with their limitations and potentialities are given in Remondino and Rizzi (2010).
In the next section, methodology steps in TLS and photogrammetry are summarized.

TLS
After a decade of continuous development, TLS systems have reached a high level of automation allowing fast and accurate surveys in cultural heritage (Grussenmeyer et al., 2010;Lerma et al., 2009).Laser scanning systems can produce data that can vary in terms of point density, field of view (figure 1), amount of noise, incident angle, waveform and texture information.Data processing procedures are complex due to edge detection, filtering, and other registrations which require dedicated software packages, high performance computers and storage capacity.Accurate texturing of point clouds is a difficult task.
However, a very low level of post-processing is required to visualize the coloured 3D point clouds and navigate through.
For the practice of TLS projects, the reader can refer to the guidelines given in (Lerma et al., 2008)  Data registration and geo-referencing steps are very important when considering point clouds from various sources.Each scanner position and orientation must be defined according to a local or global site coordinate system.An accurate georeferencing is the root of an accurate project.Direct Geo-Referencing of point clouds is possible with scanners like Trimble GX or Leica Scan Station C10.Indirect Geo-Referencing is applied when the centring and orienting options are not available in the scanner.Target-to-target or cloud-tocloud based registration are possible.For the post-processing of dense point clouds from photogrammetry, the indirect Geo-Referencing is also applied.

Photogrammetry
We focus here only on methods of generation of dense 3D point clouds and detailed surface models of physical objects from calibrated cameras, sometimes called photo-based scanning.Multiview 3D reconstruction solutions with detection of automated image correspondences using area and feature based matching algorithms are key solutions for the future of documentation.Barazzetti et al. (2010) and Ducke et al. (2011) published methodologies based on multi-image matching techniques from computer vision or photogrammetry in order to produce dense and accurate point clouds from series of overlapping images.Both academic and commercial solutions are available today for automated image orientation, surface measurement, mesh generation and texture mapping.In Hullo et al. (2009), we present a practical example based on dense stereo matching approaches for the documentation of a site in a desert where only digital cameras could be used.
The usual steps of the pipeline are the following: -Data acquisition (camera calibration, multi-view recording) -Automatic orientation of the images (relative orientation, scaling or geo-referencing, bundle adjustment) -Dense matching (stereo matching or multi-view) -Post-processing -Quality control and delivery.
-Distortion free images can be processed after calibration The main advance in the last few years has been the automation of image orientation (with or without coded targets) at a subpixel resolution.Several software packages are available, whether web-based, open source or commercial.
The accuracy of the resulting point-cloud relies mainly on the quality of the camera, the object pixel-size and of course the configuration of the field survey.

Merging TLS and Photogrammetry
Most of the scanners are today able to measure panoramas (figure 2, right).To overcome limits (holes and gaps) due to the use of hybrid scanners (figure 2, left), photogrammetry can provide complementary point clouds for uncovered areas (Alby et al., 2011b).Both TLS and photogrammetry techniques are able to deliver dense point clouds.Compared to TLS, the photogrammetric solution is a low cost one as the acquisition system is limited to a digital camera and a few accessories only.Indeed, the stereo matching process offers a cheap, flexible and accurate solution to get 3D point clouds and textured models.The main advantage of this photogrammetric methodology is to get at the same time a point cloud (the resolution depends on the size of the pixel on the object), and therefore an accurate meshed object with its texture.After the matching and processing steps, we can use the resulting data in much the same way as a TLS point cloud, but with really better raster information for textures.
In addition, point clouds from close range scans (as e.g.acquired by laser scan arms) can also be merged with TLS and photogrammetry data (see section 4).
In the following sections, examples showing different levels of integrated photogrammetry and TLS data will be discussed.

Site description and recording
The Bronze Age cave "Les Fraux" at Saint-Martin-de-Fressengeas (Dordogne, France) is made of a network of galleries of several hundred meters, used since 2008 as a field of investigation by several researchers.The cave is linked to the above terrain thanks to surveying techniques, allowing accurate geo-referencing in the French Geodetic Reference System.The aim of the documentation work (Grussenmeyer et al., 2010) is to generate a geometric and photorealistic 3D model from the combination of point clouds and photogrammetric images, for both visualization and accurate documentation purposes.As the movements in the cave are restricted along a path designed by the head of the project due to several archaeological remains all around the cave, complementary recording methods are necessary.

Instrumentation and tasks
The documentation and recording processes include different steps: -Accurate network of reference points (GPS points outside, a few marks only on paths of the cave inside), spheres and coded targets for the registration of point clouds and photogrammetric bundle adjustment; -Overall recording of all the galleries by Terrestrial Laser Scanning (FARO Photon 80, 120 or Focus 3D) at high spatial resolution (figure 3); -Overall photogrammetric recording by calibrated digital SLR cameras (Canon EOS 5D with 28mm and 85mm lenses); -Sub-millimetre scanning of clay carvings with a FARO Scan Arm; -Panoramas captured with a Manfrotto Pan Head.
We focus in this section on the 3D models of a clay panel with carvings (figures 4, 5 and 6).The data available are both TLS point clouds, Canon EOS 5D images for photogrammetry and Scan Arm data of parietal art on the clay panels  Figure 6.3D textured raw model obtained by dense stereo matching from two panorama standpoints in front of the clay panel (the distance to object is 5m, focal length 85mm, pixel size in the object is less than 0,5mm, base/distance ratio is 0,4).

Overview of results
Our experiences confirm that TLS is fast and appropriate for the overall recording of the cave but unsuitable for sub-millimetre modelling where photogrammetry dense matching and laser ScanArm can both provide very accurate models for research in parietal art.

Site description and recording
The Roman theatre of Mandeure (France) is the second biggest in Gaule.At the roman era, the city around was used to be known as Epomanduodurum.Its overall area is estimated to 2 km 2 .The theatre, leaning on a hill of the river Doubs, has a diameter of 142 meters and its capacity is estimated between 15000 and 18000 seats.Excavations have been carried out for about ten years.The main objective of the recording project consists of the realization of a digital terrain model and built elements of the theatre in its current state.Because of the size of the site and the resources available we opted for a combination of photogrammetric methods and TLS.The hybrid model is thus obtained from the fusion of different data: -Terrestrial Laser Scanner (Trimble GX) -Aerial LiDAR (from Aerodata International Survey) -Dense Stereo Matching Photogrammetry Currently, sub-millimeter point clouds of milestone fragments are recorded with a Faro Laser ScanArm.

Instrumentation -Tasks
Because of the size of the site, a long-range TLS is needed (figures 7 and 8).On the other hand, the remains of the building would have required a very large number of stations.Since, the Trimble GX TLS (standard deviation: 3mm at 50m) has a quite narrow field of view and a low scanning speed, it became obvious that this technique is not well suited for some parts of the site (the remains of the buildings and the excavations).We chose then to complete the TLS point cloud by data obtained by Dense Stereo Matching Photogrammetry (figure 9).A global geo-referenced point cloud has first been obtained by Terrestrial Laser Scanning which consists of 8 Million points obtained from 14 different stations.The spatial resolution is comprised between 1 cm and 5 cm and the accuracy is estimated to 1 cm.Another 16 million points has been obtained by dense matching.With the 28mm focal length and for a five meters distance to the object, one obtains an object-pixel size of 1.5mm.We have used standard Photomodeler coded targets to ease the orientation step and the scale setting step.Then, thanks to overlapping areas, we have merged the different projects in order to obtain « blocks » (figure 10).After registration of the "blocks" together with the TLS point cloud, they are getting geo-referenced as described hereafter.This has been achieved through the 3DReshaper ® software by means of two steps.A first coarse registration is made by selecting manually corresponding points in the two different clouds.As a second step, an automatic fine registration is being processed by means of a 7 parameters transformation (ICP algorithm) with the geo-referenced TLS cloud as the reference (figure 8).One can notice that the scale factor in this transformation has proved to be very close to unity because the clouds originating from photogrammetry had already been scaled.

Assessment of merging TLS and image based data
Visual analysis: A first quick control of the data may be done by visualizing appropriate sections as seen on figure 11.It allows to take note instantly that a gross error has been done and then to correct the process.However, in order to certify the quality of the whole process, it remains necessary to achieve quantitative control.One can notice on the histogram of the error that 57 % of the points have an error inferior to 2 cm, 66 % inferior to 3 cm and 87 % inferior to 4 cm.Some quite important errors are reported on current excavations as seen on figure 12 (parts in red).It is most likely that these errors originate from the fact that elements (blocks) may be displaced slightly during the excavation period, both clouds having not been acquired simultaneously.

Project description and recording process
The Engelbourg castle (13th century) has been destroyed in 1673 due to its situation in eagle's nest overlooking one of the major crossings between Alsace and Lorraine.These ruins have been the subject of several archaeological studies in the last years.The work of our group was to provide a 3D model to be used as a benchmark dataset for the Archaeological Information System currently under development.A combination of geodetic surveying, image matching and TLS techniques has been used (Koehl and Berger, 2011).

Instrumentation and results
The point clouds were recorded with a Trimble GX (Figure 13) and the images with a Canon EOS 5D camera.As the Trimble GX TLS has a field of view limited to 60° and a minimum distance acquisition (2m), it makes it inadequate for special situations.In this project, some narrow parts of the site have been completed by dense matching of the images with Photosynth and related toolkit (Figure 14).A geodetic network has been set prior to the recording and used for the registration of the point clouds.

CONCLUSION
The approach proposed in this paper has shown that accurate point clouds can be delivered from both TLS and dense image matching, and thus merged for documentation in cultural heritage.TLS technology is now mature and delivers point clouds easy to assess.More and more tools for dense matching of images are available (whether web-based as Photosynth, open source or commercial) with a high level of automation in the image matching and registration.
To ensure the accurate merging of the different datasets, an accurate network of geodetic control points well distributed on the study area is advised for direct or indirect geo-referencing.
In the examples discussed in the paper, the TLS available was not suited for the recording of specific types of objects and areas in the projects.TLS aided by dense image matching finally showed that a high level of integration is possible for tightly laser scanning and photogrammetry data.More examples of projects from the Photogrammetry and Geomatics group are presented in Alby et al. (2011b) andLandes et al. (2011).
and (English Heritage,  2007).The usual steps of the pipeline are the following:

Figure 2 .
Figure 2. Horizontal section of underground galleries (1m wide) recorded by TLS (left: gaps due to the use of hybrid scanner as Trimble GX, and right: better coverage with the Leica C10 panorama scanner)

Figure 3 .
Figure 3. Example of recording in the cave with the TLS FARO Focus 3D, with the overall meshed model on the right.

Figure 4 .
Figure 4. Panoramas and zoom on details of parietal art features requiring very accurate recording (object pixel size is less than 0.5mm)

Figure 7 .
Figure 7. Positioning of the Trimble GX TLS at the ground level of the theatre of Mandeure.

Figure 8 .
Figure 8. Overall view of the Gallo-roman Theatre of Mandeure (France) from the Trimble GX coloured point clouds.

Figure 9 .
Figure 9. Canon EOS 5D with 28mm focal length: stereopair (left) and 3D textured model from matching (right).500 images have been used in 2010 for the dense matching part in 125 sub-projects.The total acquisition time was about 20 hours.The local spatial resolution was about 5 to 10 mm.

Figure 10 .
Figure 10.Complete 3D point cloud of the theatre (diameter 142m) of Mandeure (France).Point clouds (in red) based on dense matching are combined with TLS data (green-blue).

Figure 11 .
Figure 11.Horizontal section of the southern part of the theatre coloured by cloud: green for the image based point cloud and blue for the TLS point cloud.Quantitative analysis: The quantitative control consists in measuring the distance between both clouds and is achieved by means of the CloudCompare software developed by the research department of EDF (France).The Hausdorff distance is computed on overlapping areas of both point clouds and the results are given hereafter.The standard deviation of the error has been computed to 2.8 cm which has been judged as being satisfying as regard to the global size of the archaeological site.One can notice on the histogram of the error that 57 % of the points have an error inferior to 2 cm, 66 % inferior to 3 cm and 87 % inferior to 4 cm.Some quite important errors are reported on current excavations as seen on figure12(parts in red).It is most likely that these errors originate from the fact that elements (blocks) may be displaced slightly during the excavation period, both clouds having not been acquired simultaneously.

Figure 12 .
Figure 12.Quantitative assessment and error map More details about this project are presented in Hullo et al. (2010) and Alby et al. (2011a).

Figure 13 .
Figure 13.Overview of the Engelbourg project

Figure 15 .
Figure 15.Point clouds merged in a panoramic view.The socalled "Eye of the Witch" is visible on the top of the hill.