GENERATING ACCURATE 3D MODELS OF ARCHITECTURAL HERITAGE STRUCTURES USING LOW-COST CAMERA AND OPEN SOURCE ALGORITHMS

These studies have been conductedusing non-metric digital camera and dense image matching algorithms, as non-contact methods of creating monuments documentation.In order toprocess the imagery, few open-source software and algorithms of generating adense point cloud from images have been executed. In the research, the OSM Bundler, VisualSFM software, and web application ARC3D were used. Images obtained for each of the investigated objects were processed using those applications, and then dense point clouds and textured 3D models were created. As a result of post-processing, obtained models were filtered and scaled.The research showedthat even using the open-source software it is possible toobtain accurate 3D models of structures (with an accuracy of a few centimeters), but for the purpose of documentation and conservation of cultural and historical heritage, such accuracy can be insufficient.


INTRODUCTION
In the last few years, the documentation of Cultural Heritage became the object of interest and study of various researchers. When working with documentation and inventory of monuments, it is crucial to be cautious and do the measurements remotely in a non-contact way.According to ICOMOS claim: "The choice between traditional and innovative techniques should be weighed up on a case-by-case basis and preference given to those that are least invasive and most compatible with heritage values, bearing in mind safety and durability requirements."There are several remote techniques for measurements, such as laser scanning and terrestrial photogrammetry.This research has been concerned using images of structures and dense image matching algorithms, as non-contact methods of creating monuments documentation.
Practical examplesthat present the applications of photographs for the 3D modelingof cultural heritage objects based on photographs are shown in (Remondino, 2008). Another work (Barazzeti, 2009) presents a combination of photogrammetric and computer vision methods for automatic modeling of terrestrial objects. There are also studies where modeling is not based only on photographs, but also on video images (Delis, 2017) or directly from 3D point clouds (Fryskowska et al., 2015). Whereas (Kersten, 2015) showed the results of usage of open-source programs in conjunction with point cloud obtained through laser scanning.
In this contribution, open-source and low-cost systems are tested with reference data from direct measurement for the complex structures of Powazki Cemetery.

The research area
The Powazki Cemetery is one of the oldest necropoleis in Warsaw. It was founded on 4th November in 1790 and was enlarged many times from that date -today it occupies 43 hectares. The entire area of the cemetery is filled withsculptures, monuments, and small architectural structures. Furthermore, the cemeteryis under the conservation protection of the Capital Conservator of Monuments. In the Powazki Cemetery, about 1 million well-known and well-deserved people, including soldiers, famous artists or scholars, were buried. Many tombstones in the cemetery are complex, and they areexceptional examples of unique sculptures, and elements of small architecture. Due to the fact that they are historical objects, their identification, management, and conservation are very important. However, the documentation of such facilities presents a challenge for various reasons. First of all, the area of the cemetery is very large (over 40 hectares). Secondly,the number of individual gravestones and individualmonuments is close to several hundred.Therefore an automatic and fast inventory method is desirable. An additional problem could be the location ofgravestones -they are often located close to each other, so the quick and easyinventoryfromall sides is not possible. Moreover, there are many plants or trees covering the same parts of structures.
For the research purpose, three tombstoneswere selected. The first wasa magnificent tomb with a complicated structure, resembling a chapel with a bright facade, the second one -was a gravestone with an angel statue made of stone, and a thirdwas aquite low tombstone with a visible metal roof and a stone pedestal. All of them are showed in figure 1.

Description of used tools and software
All images of selected objects were made with digital compact camera Kodak EasyShare C613 with a 6Mp matrix, a focal length of 6 mm and an aperture of f /2.7. Photographs were taken without a flash, with general overcast to avoid shadows on objects. For each investigated object different number of images were taken-47 images for the first tombstone, 43 for the secondone and 53 for the thirdone. In order to generate point clouds and 3D models from the acquired images, open source VisualSFM, OSM Bundler/PMVS2 and ARC3D webservice packages have been used. All three solutions are generally open source and free for non-commercial use. The SfM algorithms vary in their features and options. Some software (especially open source) resamples images to speed up calculations. Thus, a high-resolution camera is not usually required, but this might limit the accuracy of generated model.
Open-access software CloudCompare has also been used to filter clouds and generate textured models.

Generating 3D models
Structure-from-motion technique was used to reconstruct 3D objects on the basis of a vast number of images. The SfM is the process of reconstructing the3D object from its projections into a sequence of images taken from different perspectives Schönberger, 2016). This technique differs from traditional photogrammetric approaches by determining internal orientation parameters and camera position, routinely and without the need for a predefined set of ground control points (with known three-dimensional position). Incremental SfM is a successive processing with an iterative reconstruction component. It normally starts with feature extraction and matching, and next goes with geometric verification. The resulting scene graph serves as the basis for the reconstruction step, which selects the model with a two-view reconstruction. Then algorithm registers new images, triangulates scene points, filters outliers, and refines the reconstruction using bundle adjustment. Limitation of this solution isa need for a high degree of overlap to cover the full geometry of the object or scene of interest because a large number of corresponding points must be defined in each view. Additional complication is that there occur different kinds of degenerate structure and motion configuration for which the standard algorithms will fail (camera rotation without the translation, planar patches on the scenes, irregular lighting or points lying on a line passing through the optical centers of the cameras in which it is visible). In practice, it may be hard to avoid these kinds of defects, particularly if non-expert user obtains images. and compared. The scale value has been determined and averaged; then it has been applied in thesoftware. Based on these point clouds, 3D textured models using the Poisson Plugin Reconstruction in CloudComapre were generated. In the figure2, each stage of all process is shown.
3D models have also been madewith the ARC3D web application (Vergauwen, 2006), but in this case, the user's interference in the whole process is quite limited. The 3D reconstruction is based on theprinciple of auto-calibration, feature detection and correlation, dense multi-stereo reconstruction and point cloud generation. The ARC3D software has been developed to compute the reconstruction over a distributed network (cloud) of computers.The user sends a series of images to the server, once the reconstruction is successful, the system notifies the user by email. The modelsobtained with such methodwere also scaled in CloudCompare.

RESULTS
Finally, two point clouds and three textured 3D models of objects have been generated. All models differ in their completeness. The visual analysis of theeffects showed that results obtained with VisualSFM and OSM Bundler are similar, while products obtained with ARC3D do not reconstruct the entire photographed structures. Although texture quality in the third case is much better, the reconstruction results are not satisfactory because many elements of the objects are missing. Moreover, the discrepancybetween generated cloud points has been verified. Figure 3 shows deviations between individual point clouds. With the red color, the mean differences of more than 6 cmare shown, the difference of a few millimeters is highlighted in blue. It is confirmed that the VisualSFM and OSM Bundler programs produce similar results, and point clouds generated by them vary on average by several millimeters. Clouds received via ARC3D are incomplete, so the red color infigure 3 indicates no points in the cloud. On every cloud and mesh, reference distances were measured, as illustrated in Figure 4, and the RMS errorswere calculated on this basis.Over 20 different dimensions for each object were examined, and residuals were checked. The results of the calculations are given in Table 1. The discrepancies between the point clouds and the 3d models are in the range of 2-6 cm. In most cases, larger RMS errors have been generated for cloud points than for models created from them, due to the difficulty of pointing to the correct reference points. Moreover, the best results have been obtained with the OSM Bundler point clouds and the models generated from them. On the other hand, the ARC3D software is the worst in this comparison -in this case, it was not possible to measure some dimensions due to incomplete models.

COMPARISON OF OBTAINED RESULTS
The research shows that even when the average quality camera and free softwareare used, it is possible to produce fairly accurate and realistic looking 3D models. The accuracy of such models relatively tooriginalstructures is about several centimeters. However, it should be bare in mind that RMS errors were affected not only by the surface reconstruction process but also by the model scaling process and the precision of the operator when reference points were picking.
The OSM Bundler software, which produced the most accurate point clouds using the PMVS2 algorithm, was the best. Definitely, ARC3D did not meet the expectations, models obtained through it are better visually (they are better textured) than others, but do not meet the requirements of completeness, despite many attempts failed to get results that would represent the selected objects in its entirety.

CONCLUSION
At low cost and in relatively short time, it is possible to create spatial models of historicstructures that can be used for general visualization and eventual recording of objects. Unfortunately, automatically generated models do not provide the accuracy required for architectural documentation of monuments.Free software is especially sensitive to the different lighting conditions at the time of taking imagesorthe vegetation that is obscuring thescene and preventing direct access to objects.In the case of a few centimeters decorative details, such solutions do not provide areconstruction of perfect shape.Nevertheless, separate modeling of individual elements (not the object as a whole) is possible. Considering the pace of changes in the software market, can we expect, that shortly open-source solutions could satisfy the current accuracy expectations and would contribute the efficiency increase of the entire modeling process. Figure 5. Resulting 3d textured models: left column: VisualSFM/CloudCompare, middle column: OSM Bundler/CloudCompare, right column: ARC3D