INTEGRATION OF POINT CLOUDS FROM TERRESTRIAL LASER SCANNING AND IMAGE-BASED MATCHING FOR GENERATING HIGH-RESOLUTION ORTHOIMAGES

An orthoimage is one of the basic photogrammetric products used for architectural documentation of historical objects; recently, it has become a standard in such work. Considering the increasing popularity of photogrammetric techniques applied in the cultural heritage domain, this research examines the two most popular measuring technologies: terrestrial laser scanning, and automatic processing of digital photographs. The basic objective of the performed works presented in this paper was to optimize the quality of generated high-resolution orthoimages using integration of data acquired by a Z+F 5006 terrestrial laser scanner and a Canon EOS 5D Mark II digital camera. The subject was one of the walls of the “Blue Chamber” of the Museum of King Jan III’s Palace at Wilanów (Warsaw, Poland). The high-resolution images resulting from integration of the point clouds acquired by the different methods were analysed in detail with respect to geometric and radiometric correctness.


INTRODUCTION
Cultural heritage has become a domain of wide popularity for close-range photogrammetric techniques.High-resolution elaboration of difficult-to-access objects which may be easily destroyed is not difficult if non-invasive photogrammetric measuring technologies are applied (Chiabrando, Spano, 2013).Conventional measuring technologies, such as tacheometry, are often substituted by photogrammetric data acquisition methods because they are time consuming and usually do not guarantee the expected level of detail (Vosselman, Mass, 2010;Grussenmeyer et al., 2008).The use of close-range photogrammetry for works related to cultural heritage still needs further research.The main research strand aiming to optimize modelling of historical objects focuses on the integration of data collected by means of several measuring technologies (Guarnieri et al., 2006;Ramos, et al, 2015;Markiewicz, Zawieska, 2014;Markiewicz et al. 2015).Combining of data from terrestrial laser scanning and image matching is increasingly being used to improve the quality of products (Baptista, 2013).Integration of two measuring technologies can result in the elimination of the weaknesses of one by the advantages of the other.

METHODOLOGY OF DIGITAL ELABORATION OF HISTORICAL OBJECTS
The following measuring techniques have recently been those most commonly used for documentation of historical objects: terrestrial laser scanning (TLS), and photogrammetric techniques based on image matching.Both measuring technologies require adequate photogrammetric knowledge in the data acquisition phase as well as in post-processing.(Van Genechten, 2008;Gianolio, 2014).
Successive stages of data processing, both in TLS and in image matching, are mutually dependent.If any stage is omitted, it may result in generation of a photogrammetric product which does not meet expected accuracy requirements (Bornaz, Rinaudo, 2004).

Terrestrial Laser Scanning
The growing popularity of terrestrial laser scanning for investigations of historical objects results from the possibility to acquire three-dimensional clouds of millions of points within a very short time.The unquestionable advantage of this technology is its high accuracy and the high level of automation of measurements (Bastonero et al., 2014;Fryer et al., 2007).However, such big datasets are difficult to process quickly or to integrate with GIS-type systems (Aitelkadi et al., 2013).Apart from angular-and-linear measurements, the scanner also records the laser-beam reflectance intensity in order to investigate properties of the analysed object (Buckley et al., 2012).Despite its high popularity, the terrestrial laser scanning method also has some disadvantages, which are mainly connected with instrumental errors.The laser beam propagates with distance, which can result in uncertainty regarding the location of the determined point (Van Genechten, 2008).Another source of errors in TLS measurements is reflectivity of the investigated objects.The laser beam is fully absorbed by elements of too-low reflectivity; this results in data gaps within the point cloud.
In the case of high-reflectivity surfaces, the reflected laser beam does not reach the detector, and therefore the locations of points are incorrectly determined (Markiewicz, Zawieska, 2014).

Image matching
Collection of 3D information from photographs is possible by means of measuring homologous points in images in which the same object has been photographed from different places.Dense image matching allows full automation of this process (Chiabrando, Spano, 2013).This method of measurement consists of recognition of similar features (primitive) in pairs of images.Depending on the feature type, two types of matching exist: -Area-Based Matchingwhich is based on the images' optical density (Cyganek, Siebert, 2009); -Feature-Based Matchingwhich searches for corresponding details (features) in images (Lowe, 2004;Bay et al., 2006).
Generation of dense point clouds based on a group of photographs is possible using the Structure from Motion algorithm combined with the Multiview Stereo approach, which forms the foundation of modern photogrammetry.In this method, 3D reconstruction of an object is performed automatically based on corresponding valid geometric features across images, acquired with sufficient overlap.The accuracy of the resulting point cloud is influenced by, among other things: the base-to-depth ratio, the number of acquired photographs, the number of measurements of photographs, and the resolution of photographs (Moussa, 2014;Markiewicz, Zawieska, 2014).The main limitation of the image matching method emerges with objects of uniform texture: for such objects, it is almost impossible to find homologous points.

Integration of TLS and image matching data in elaboration of historical objects
The growing accuracy requirements of photogrammetric elaborations of historical objects have started a debate concerning the possible integration of different measuring data (Bastonero et al., 2014).Historical objects usually consist of elements of different textures, shapes, and materials.Laser scanning performs 3D measurements of such objects within a short time, but does not allow correct determination of the locations of all details.Dense point clouds may complement point clouds obtained from TLS (Altuntas, 2015).The goal of this experiment is to present methods of integrating point clouds from terrestrial laser scanning with dense cloud points in order to increase the quality of generated images.

EXPERIMENTAL SECTION
The subject of the work was the interior of one of the chambers at the Museum of King Jan III's Palace at Wilanów, the so-called "Blue Chamber" (Figure 2).During the first stage of the work, survey marks were designed and distributed.These were considered as the photogrammetric control, being the base for orientation of scans acquired from different places, as well as for the image bundle adjustment process.Nine printed survey marks were designed and placed on the walls of the Blue Chamber.Point clouds were acquired by a Z+F Imager 5006h terrestrial laser scanner with "super-high" scanning resolution, corresponding to 3.2 mm between points for every 10 m of scanning distance.Twelve scans were acquired from different scanner stations.This number of measuring stations was determined by, among other things, the location of the gilded elements close to the ceiling or the convex fireplace on one of the walls.Series of photographs of the entire chamber were acquired by a non-metric Canon EOS 5D Mark II camera with a wide-angle lens of 20 mm focal length.The camera was equipped with a small-size CMOS array (36 mm x 24 mm) of resolution 21 M pixels.About 150 photographs were acquired of resolution 5616 x 3744 pixels recorded in .cr2format, the original file format of RAW images.The field resolution of a pixel (GSD) was equal to 0.8 mm.

TLS point clouds processing
Point clouds were acquired from each scan in a local system; then, mutual orientation of all sets of points was performed, i.e., particular point clouds were transformed into a unified reference system.The local system, connected with the scanner station in the centre of the investigated object, was taken as the reference system.The performed mutual orientation was accurate to 1.6 mm, with a standard deviation of 1.1 mm.The results were considered correct with respect to the accuracy of angular and linear measurements that was achievable using the laser scanner Z+F Imager 5006h.The error of distance measurements did not exceed 1 mm, while the error of angular measurements (after recalculation as arc measurements), taking 5 m as the sight distance, was 0.6 mm.The next stage consisted of filtration of the point cloud, which considerably improved the data quality.The distance filtration and the intensity filtration were performed, which resulted in elimination of noise recorded in the air, points of the highest and the lowest reflectivity, and points registered outside the area of investigation.

Image processing and dense cloud generation
Orientation of photographs was performed using Agisoft PhotoScan Professional software.The Structure from motion algorithm was used to determine the exterior orientation for each photograph and to generate the sparse point cloud.The next stage was generation and visualization of the dense point cloud, created as a result of automatic image matching.Based on the camera position, estimated earlier, the software calculates the so-called depth information for each photograph, which is then used to connect single points of the cloud.The basic parameter to be determined directly before the pointcloud generation process was the model's expected quality.The selected parameter directly influenced the accuracy of the generated point cloud, and thus the accuracy of the derivative product, i.e., the orthoimage.Exemplary point clouds were generated assuming two highest values of the quality parameter, i.e., high and ultrahigh (Figure 3).The point cloud generated under the assumption of high quality looks better (Figure 3B).It does not have many blind spots; however, correct generation of points was problematic on flat and almost homogenous surfaces.In relation to geometric correctness, the cloud B) in Figure 3 artificially filled the blind spots on the walls using the nearest-neighbour method.With relatively big distances between edge points, this method cannot be considered appropriate for interpolation.Besides, the point cloud with ultrahigh quality (Figure 3A) allowed better results in connecting flat elements, such as the blue wall and the window-sill.As a result of the integration of point clouds acquired from image matching with TLS data it was possible to replace fragments of clouds which visualized gilded elements, thus achieving higher accuracy.The fragment of decorations shown in A) in Figure 3 is characterized by a higher level of detail and geometric correctness.

Comparison between orthoimages generated from TLS point clouds and dense image-based clouds
LupoScan Pro software was applied to generate orthoimages.In the first step, a reference plane was determined, which served as the basis for generation of the Digital Surface Model (Markiewicz et al., 2015) in the GRID structure.The nearest-neighbour interpolation algorithm was used for this purpose.Additionally, the intensity value from the TLS data (Figure 4A) and the RGB colour image based on image data (Figure 4B) were generated for each node of the GRID network.Known information about earlier-oriented digital images allowed backward orthorectification.Orthoimages were the subject of detailed visual evaluation and accuracy analysis.
In the case of wall paintings, a lack of data caused by the uniform texture is clearly visible in the orthoimage gained from image matching (Figure 5B).TLS correctly projected objects with unified textures (Figure 5A).Gildings on the ceiling were characterized by high reflectivity valuesdata were missing here in the TLS point cloud.In the TLS orthoimage, gilded elements seemed blurry and noisy (Figure 5C).Image-based matching provided satisfactory results for under-ceiling gildings they were sharply visualized in the orthoimage (Figure 5E).Connections of convex elements with a flat wall were correctly projected in the TLS orthoimage (Figure 5E).In the orthoimage from image matching, noise is visible, caused by the low quality of the point cloud within this area (Figure 5F).In the case of the Blue Chamber, the use of only one measuring technology did not produce satisfactory results.Therefore, it was reasonable to combine TLS data with image matching in the generation of high-resolution orthoimages.

Integration of point clouds from terrestrial laser scanning and image matching
The methodology of data integration, aiming at improvement of the resulting product, consisted of generation of an orthoimage based on two independent point clouds: one from TLS and one from image matching.The process of integration is schematically presented in Figure 6.Two 3D sets of points, from TLS and image matching, respectively, were projected on a defined plane.Then, backward rectification was performed (Figure 7E).The obtained photogrammetric product was characterized by visible improvements in the quality of the projected under-ceiling gildings (Figure 7A) compared to the intensity orthoimage.The edges of every gilded element are clearly visible, which would support possible orthoimage vectorization.Therefore, the positive impacts of densification of the dataset with the point cloud obtained from image matching may also be noticed.Unfortunately, the resulting orthoimage is also characterized by numerous imperfections when transferring from convex to surface objects.Examples of such errors include the edges of the shutters (Figure 7B) and the gilded wall elements (Figures 7C, 7D).Acknowledging the limitations of the technology of image matching, the image matching point cloud may be applied differently.E) It was proposed to limit the point cloud to the area where it is necessary to densify the 3D point dataset.In the case of the Blue Chamber, the point cloud was limited to the fragment of the under-ceiling gildings (Figure 8).The approach to integrate point clouds in order to increase the orthoimage accuracy did not change.Instead of the entire point cloud from image matching, only the fragment was used.The 3D point dataset was therefore densified only in the area where the point cloud from TLS contained the highest noise.
The resulting orthoimage is presented in Figure 9.
The resulting orthoimage is characterized by high accuracy, both for complex gildings and simple elements.The use of only a fragment of the point cloud allowed the elimination of the negative impact of data integration on the correctness of the projection of elements of uniform texture.The integration of point clouds from TLS and from image matching gave a photogrammetric product which meets the accuracy requirements of the investigation of historical objects.

SUMMARY
In the investigation of historical objects, it is important to achieve high accuracy in the final products.Unfortunately, due to limitations during measurements performed with different sensors, integration of different measuring techniques is required.Limitations of measurements performed with the use of the terrestrial laser scanner emerge in the process of the projection of under-ceiling gildings (the influence of the big scanning angle and the surface reflectivity), as well as gilded decorative elements.Unfortunately, using the SfM technique combined with the MVS, it was not possible to correctly project flat surfaces of the uniform structure.Despite the change in the sampling resolution, the wall surface was not correctly projected.Integration of data acquired by means of two technologies may be the solution to these problems.Simultaneous use of point clouds of different natures allowed correct projection of all elements of the historical object.Limitations of terrestrial laser scanning connected with the reflectivity of a registered object were eliminated by densification of a 3D point dataset with data from a cloud acquired from image matching.In summary, the growing accuracy demands of the investigation of historical objects are a good motivation to integrate multi-source data.The obtained results prove that the use of TLS and image matching techniques effectively complement each other, in particular in the case of complex historical surfaces with rich decorations.In many cases, such an approach may be the only method to develop the required photogrammetric products with respect to both geometric and radiometric aspects.

Figure 2 .
Figure 2. A section of the "Blue Chamber" at the Museum of King Jan III's Palace at Wilanów (Warsaw, Poland).

Figure 3 .
Figure 3. Point cloud generated from image matching and zoomed gilded elements: A) ultrahigh quality; B) high quality.

Figure 4 .
Figure 4. A) Orthoimage from reflectance intensity.B) Orthoimage generated based on the point cloud from image matching.

Figure 5 .
Figure 5.Comparison of examples of elements of an object on the orthoimage from TLS and from the image matching point cloud.

Figure 6 .
Figure 6.Diagram of integration of point clouds from terrestrial laser scanning and image matching in the generation of high-resolution orthoimages.

Figure 7 .
Figure 7. E) An orthoimage generated from integration of point clouds from TLS and image matching.(A-D) Zooms show selected elements of the processed wall.

Figure 8 .
Figure 8. Limiting the point cloud from image matching to under-ceiling gildings.

Figure 9 .
Figure 9. A) An orthoimage generated from integration of point clouds from TLS and image matching.B) Zooms show selected elements of the processed wall.