INVESTIGATION INTO THE USE OF ACTION CAMERAS IN THE DOCUMENTATION OF ARCHITECTURAL DETAILS-THE CASE STUDY OF A BAROQUE CHAMBER

In this paper the possibility of using the Xiaomi 4K action cameras as a low-cost sensor for the generation of high resolution documentation of architecture and architectural elements in the field of Cultural Heritage was analysed. For that purpose a series of images was acquired together with tachometric measurements to determine the ground control points. Additionally TLS data was collected, which was treated as a reference. For the purpose of point cloud generation the Structure-from-motion (SfM) and MultiView Stereo (MVS) approaches were used. The following parameters of the collected data and the resulting documentation were tested: the interior orientation parameters analysis, quality of the Xiaomi built-in Lenses Distortion Correction; the accuracy of the orientation on ground control and check points, the point cloud density; the flatness of the walls; the discrepancies between point clouds derived from the low-cost cameras and TLS data, shape of the architectural details based on cross-section analysis. After the analysis of the obtained results it can be concluded that the Xiaomi 4K low-cost sensors are well suited for the purpose of documentation of architecture and architectural details. All the data for the presented investigation were acquired at the baroque residence of the Bieliński Palace in Otwock Wielki in Poland.


INTRODUCTION
The use of documentation obtained through laser scanning and photogrammetric approach is a well-established practice in all fields dealing with architecture and architectural details such as Art History, Archaeology, or Cultural Heritage Management and Preservation.This type of documentation can be gathered with the use of different sensors, such as a wide array of cameras available to investigators or lasers scanners.
Generation of precise and, at the same time, high resolution architectural documentation, acquired on the basis of terrestrial laser scanning or dense point clouds from digital images is a still open issue, which presents many challenges.Although data processing algorithms are intensively developed with consideration of high resolution orthoimage processing or generation of accurate vector drawings, limitations concerning the utilisation of measuring sensors are still the issue which should be solved.
The choice of the sensor is usually determined by the needs of the specific project and the funds available.There is a general consensus that the higher the resolution of the sensor the higher the quality of the final documentation.In the presented paper we investigate the possibilities of using low-cost sensors, in our case the Xiaomi 4K action cameras for the documentation of complex architectural details with the application to Cultural Heritage Management and Preservation.In order to gain an insight into the metric properties of the documentation we compare it with the results obtained through the use of a phasebased laser scanner Z+F 5006h (treated as a reference data).
In order to validate the possibility of using Xiaomi 4K low-cost cameras in Cultural Heritage documentation, following parameters were analysed:  the interior orientation parameters analysis,  quality assessment of Xiaomi camera built-in Lenses Distortion Correction,  the accuracy of the orientation on ground control and check points,  the point cloud density analysis,  the flatness of the walls,  the discrepancies between point clouds from the low-cost cameras and TLS data,  shape of the architectural details based on cross-section analysis.
The results presented in this article are the outcome of the investigation by the interdisciplinary team of researchers from the Warsaw University of Technology and the Systems Research Institute of the Polish Academy of Sciences, carried out at the Bieliński Palace in Otwock Wielki in Poland.This investigation is a continuation of previous works on the use of close-range photogrammetry in fresco documentation (Markiewicz, et. al., 2017).There following problems were discussed: the possibility of using the low-cost Xiaomi cameras (2K and 4K) in frescoes documentation, the quality of the point cloud based on flatness analysis, and comparison of point cloud obtained from Xiaomi with the point cloud from full-frame camera.After the analysis authors have chosen the Xiaomi 4K camera for the following experiment because of its higher quality.In this article the test field is characterized by complex shapes such as decorated fireplace with two antithetic figures of winged female sphinxes and tondo in a decorative stucco frame filled with a now damaged fesco.

STATE OF THE ART
The development in the field of computer vision makes it possible to generate a photogrammetric documentation using an unordered set of digital images.As a tool for Cultural Heritage, photogrammetry is appreciated for being relatively easy and fast, requiring little additional work and tools when compared to traditional methods.The final product in the form of orthoimages allows for additional measurements, not recorded on-site, which is of great importance in the investigation of architecture (Gianniou et al., 2007;Georgopoulos et al., 2004Georgopoulos et al., , 2005)).The quality of the entry data is of great importance for the quality of the final product, however as it will be shown below, low-cost sensors are a widely available source of images producing reliable results.Additionally, there is a growing number of commercial as well as open-source softwares for processing sets of images and generating orthoimages, making photogrametry a common choice in a wide array of fields (Remondinio el al., 2014, Kersten el al., 2016).The final product and the processed data from these softwares, especially point clouds, can be an important staring point for further analysis (such as angles of structures and surface flatness), which allows for the analysis of the geometrical properties of the documented structures.
Low-cost sensors such as action cameras are becoming easily available, however their quality, especially less stable lenses, render a calibration step necessary.In order to address this necessity many cameras are equipped with built in selfcalibration system.By self-calibration is we mean a calculation of the inner orientation parameters, during the adjustment process including the calculation of the object point coordinates and external camera orientation parameters.This method is most effective for digital images (Kraus, 1997;Clarke and Fryer, 1998;Cardenal et al., 2004).The differences in calibration models was widely described by i.e. (Brown, 1971, Zhang and Yao, 2008, Wang, 2012).

Test field description
The subject of our investigation is one of the rooms of the Bieliński Palace, near Warsaw (Poland), decorated with frescoes covering all walls of the room and an elaborate Stucco decoration crowning a fireplace.
The Palace is located 30 km south from Warsaw, on an artificial island in the oxbow lake of Vistula River.The main part of the palace was built in the 80's of the XVII century, and later in the 40's of XVIII wings were added.The Palace is a great example of late baroque noble residence.
The investigated room is located on the upper floor of the Palace, in the SE corner of the central part of the palace.The fireplace is located on the W wall.Its whole decoration is executed in the stucco technique.The fireplace opening is framed by a simple cornice, flanked by spirals.Above the cornice, on a simple ledge rest two antithetic figures of winged female sphinxes (Fig. 1B).On their backs rests a painted fresco tondo in a decorative stucco frame, flanked in the upper part by putti.In our investigations we focus on the figures of the sphinxes (Fig. 1C).The precise details on their wings, as well as the hair and the laurel wreaths, allow us to test the accuracy and the details of the point clouds generated.

The description of used equipment
In the conducted experiment a low-cost Xiaomi 4K camera, Z+F 5006h terrestrial laser scanner were used as well as a Total Station for measuring ground points.
The Xiaomi Yi 4K camera has a low-cost sensor with a 4000×3000 pixels resolution, the focal-length of 2.68 mm and the pixel size of 1.55 µm.It has a wide angle ("fish-eye") lens, which influences the radial distortion on the edges of the images.For the purpose of our experiment the camera was mounted on a horizontal bar in order to get a stereoscopic type camera, additionally external light sources were added.
Point clouds in the investigated interior were obtained using a Z+F 5006h terrestrial laser scanner.The resolution of the acquired data was 6 mm/10m.The maximum angle range of the point clouds was 310/360 degrees.The distance between the scanner position and the object was between 1.5 to 8 m.
In order to determine the ground control points the Leica TCRP 1202 total station was used, which allows to measure angle with 2" accuracy and reflectorless distance measurement with 2 mm +/-2 ppm accuracy, respectively.As a result of the observation adjustment the accuracy of 1.5 mm for control and check points was obtained.

Photogrammetric measurement
The measurements begun with the establishment of the 3D control network.The control network was the base for the orientation of the images and comparison of the TLS data with image products (dense point clouds).On each wall few points were measured (at least 6 points on every wall) (Fig. 2A-C).
The points should be easily identifiable and distinguishable both on the images and the TLS scan.Additionally, the natural control points should be evenly distributed on every wall.The control points were measured with the use of tacheometry with non-prism distance measurement (Fig. 2B).Simultaneously X, Y and Z coordinates of the measured points were determined.
The established control network determines the local coordinate system for the images and TLS data.
After the tacheometric measurements of the control and check points were done, 404 images of the room were taken with Xiaomi Yi 4k camera (Fig. 3A).Further, measurements with the terrestrial laser scanner were conducted.All scans were obtained from similar position (Fig. 3B).

Pre-calibration and self-calibration
The Xiaomi Yi 4K camera calibration was conducted using the Agisoft Lens, which is an automatic lens calibration software.
In this camera calibration approach the computer screen was used as a calibration target.The Xiaomi Yi 4K camera is equipped with Lens Distortion Correction (LDC), Therefore, in the first step 15 distorted images of the calibration target were acquired.For radius equal 1500 pixels, the distortion reached 90 pixels (Fig. 4A).Further 19 undistorted images, with the usage of integrated LDC were taken.For radius equal 1500 pixels, the distortion was between 4 and 5 pixels (Fig. 4B).The results of the calibration shows that the built-in algorithm for image distortion correction, reduces the influence of "fisheye effect" correctly (Fig. 4).Additionally, the parameters of distortion distribution occurring during the self-calibration process were analysed (Fig. 5).

Figure 5. The example of distortion distribution after selfcalibration process on undistorted images
The analysis of Figure 4 proves that the process of selfcalibration was done correctly.The distortion effect is the biggest on the edges of the images and its distribution is symmetrical, which proves the correctness of the parameters obtained.It should be noticed that this means that this kind of low-cost devices are appropriate for the architecture documentation.

Image orientation analysis
In order to check the quality of image orientation the SfM (Structure from Motion) approach implemented in Agisoft PhotoScan has been used.As reference data, the ground control (20) and check (10) points from the geodetic measurements were used:  the RMSE reprojection error was about 2.3 pix,  the RMSE on control points for X value was 3.4 mm, Y value 3.5 mm and Z value 2.1 mm,  the RMSE on check points for X value was 5.8 mm, Y value 2.9 mm and Z value 3.3 mm,  the RMSE related to pixels for control was 0.6 and 1.0.
The obtained results shows that quality of the images orientation is very high, so that those cameras might be used in the process of monuments documentation.

Point density analysis
One of the most important factors, affecting the quality of point clouds is their density.After conducting the orientation of the acquired images, dense point cloud were generated.Two settings of point cloud density were chosen in Agisoft software: High and Ultra High.For each wall of the room densities of the point clouds were analysed, as well as the flatness of the relief stucco decoration and frescos were examined.The mean density of the dense point cloud was approximately 30 points per 10 cm x 10 cm (Fig. 6) cell, but in some places, where the fireplace and bas-relief occur, the point density was higher.

Plane flatness analysis
In the course of the flatness analysis, for chosen architectural elements planes were fitted and differences between the point clouds and the fitted planes were calculated.For that purpose an application in MatLab based on function pcfitplane has been created which allows to automatic fitting of a plane.The main parameters of this function are planeModel object specified as a 1-by-4 vector and maxDistance which determined the distance from an inlier point to the plane.The four parameters [a, b, c, d] describe the equation for a plane: ax+by+cz+d=0 are computed.This function uses the M-estimator SAmple Consensus (MSAC) algorithm to fiting the plane.Figure 7 shows the accuracy of the plane fitting into the SfM point cloud generated from the images taken with the low-cost cameras.The distribution of errors, based on map deviations from the plane, was analysed.It might be noticed that the deviation for Xiaomi 4K MVS point cloud is lower than +/-12 mm (Fig. 6) and the histogram of the acquired values is similar to the Gaussian (normal) distribution.For all of the cameras deviations are similar and about 95% of the points do not exceed deviation of 0.008 m.
The examined walls have some geometric irregularities because of their historical character.Additionally, on the top of the walls a large deviation might be noticed.This is due to the fact that these parts were covered by a lower number of images and are characterized by worse texture than the rest of the wall.In Figure 7A and D the areas with the incised straight vertical lines were shown, which are the sketches used for frescoes painting.Some deviation might also occur because of the degree of damage of the walls and their partial renovation.

Deviations between Xiaomi 4K and TLS point clouds
The geometric quality of the surfaces obtained from laser scanning was compared with the quality of clouds of points generated from digital images.One of the most important factor which influences the quality of final documentation, such as orthoimages or 3D models, is the accuracy of point cloud.In order to check the accuracy of projection of the sculptures, the differences between the point clouds from Xiaomi 4K and TLS data were analysed.Figures 8 -9 shows the results of the comparison.
Figures 8 show that the distribution of differences between the two point clouds are similar for all of the directions (X, Y and Z).The most significant deviations are on curved parts and did not exceed +-8 mm (Fig. 8D).However, the highest deviations occur for X (Fig. 8A) axis and the lowest on for Z direction (Fig. 8C).
In the case of the sphinxes figures the mean deviations for X was equal to 0.0 mm, Y -0.0 mm and Z -0.0 mm.In table 2 was shows the percentage number of points with specified deviations Table 1.Percentage number of points for specified deviationssphinxes figures.

Deviations
[mm] -12 : -9 0.5 0. To summarize, the Xiaomi point cloud allows to generate a 3D model of complex sculptures with accuracy higher than 6 mm.Similar analysis was performed for fireplace frame.Figure 9 show that the distribution of differences between the two point clouds are similar for all of the directions (X, Y and Z).As well as in the previous test field, the highest deviations occur on curved parts and on those without good, distinguishable pattern and does not exceed +-12 mm (Fig. 9D).The highest deviations occur for Y axis (Fig. 9B) and the lowest on for Z direction (Fig. 9C).In the case of the decorated fireplace the mean deviations for X value was equal to 0.1 mm, Y -0.1 mm and Z -0.0 mm.The result of a detailed analyse was shown in Table 2.
To summarize, the Xiaomi point cloud allows to generate a 3D model of fireplace with accuracy higher than 12 mm  The analysis process also covered point clouds acquired for less pronoun relief stucco decorations of complicated geometric features (Fig. 9 E-F).Results of analyses are presented in Fig. 9 E-F; the mean deviations for X value were equal to 2.3 mm, Y -1.1 mm, Z -0.0 mm.In table 3 the percentage number of points with specified deviations was shown.
To summarize, the Xiaomi point cloud allows to generate 3D model of sculpture with accuracy higher than 6 mm.

Shape analysis -cross-sections
In order to perform an independent analysis of the shape reconstruction, the cross-sections for previously mentioned test field were verified.On the Figure 10 an example of vector drawing was show.The cross-sections based on TLS data was marked as a blue polyline and the cross-sections from Xiaomi point cloud as a red one.In the case of more complex shapes it might be noticed that a Xiaomi vector drawing is generalized and some of the details might be lost (Fig. 10A).For the second test field (Fig 10B ) it can be observed that for rounded shapes (upper part) the shape is similar to the shape approximated from the TLS point cloud.Unfortunately, similar to the first test site (Fig. 10A), the shape was not correctly approximated and maximal deviation are about 20 mm.In the case of the third (decorative stucco frame), which is characterized with the simple shape the deviations between the cross-sections are the lowest (Fig. 10C).

CONCLUSIONS
As a result of the conducted analysis we feel confident to positively evaluate the possibility of using Xiaomi 4K cameras in the documentation of Cultural Heritage objects, especially architecture and architectural elements.The achieved errors of the finished documentation are less than 1 cm for objects with simple geometry.From the wall flatness analysis it result that for up to 95% of points the distance from the fitted plane is no more than 8 mm.During the process of images gathering it was necessary to provide uniform lightning to the documented surfaces in order to avoid problems during orientation and generation of the point cloud.
For more complex shapes, the obtained data was compared with data gathered through TLS.The performed analysis shows that the error in the shape rendering for up to 95% of points in no greater than 9 mm.In the case of the frame of the fireplace, which is characterized by a texture which is difficult to process, as it is smooth and has very little characteristic points, the generated point cloud had the accuracy no greater than 12 mm.
When using Xiaomi 4K cameras in the process of documentation it is crucial to remember that the lens is a fisheye type lens.From the conducted experiment it results that the distortion occurring on the image's edges can be correctly removed during the self-calibration process.Additionally, the average point cloud density was 30 points/dm 2 .
All of the above results justifies the use of Xiaomi 4K cameras in the documentation of Cultural Heritage monuments, despite them being a low-cost device.However, it has to be taken into consideration that with the rising complexity of the documented object the accuracy of the documentation decreases, and that the process of data gathering requires specific conditions to be satisfied, such as a uniform and sufficient lighting.
To conclude, we believe, that in some aspects of the process of documentation the Xiaomi 4K can successfully substitute professional devices, and be an efficient low-cost alternative for them.

Figure 1 .
Figure 1.The example of decoration used in the process of low-cost Xiaomi 4K camera validation.A) N Wall with frescoes, B) Decorated fireplace with two antithetic figures of winged female sphinxes on the W wall, C) Fresco tondo in a decorative stucco frame above the N door.

Figure 2 .
Figure 2. The distribution of control and check points used in TLS and images orientation.A) Sketch of point distribution, B) The example of natural measured point, C) The example of marked points on dense point cloud.The room is entirely decorated with pale-coloured frescoes running down from the ceiling featuring landscapes with elements of architecture.The room has three doors, one in the centre of N wall, and the and two in the SE corner.The door in

Figure 3 .
Figure 3.The sketch of A) Image distribution, B) TLS positions 4. RESULTS

Figure 4 .
Figure 4.The sketches of the distortions: A) before B) after build-up Lense Distortion Corrections (LDC).Vertical red linethe half of the sensor size.

Figure 7 .
Figure 7. Map of MVS point deviation from a fitted plane for 4 analysed walls

Figure 9 .
Figure 9. Deviations between point cloud form TLS and Xiaomi 4K camera for fireplace: A) X value, B) Y value, C) Z value and D) linear vale and decorative stucco frame: E) X value, F) Y value, G) Z value and H) linear vale.Scale bars for X, Y, Z values (I) and linear deviations (J)

Figure 10 .
Figure 10.The example of cross-section.Red -Xiaomi point cloud, Blue -TLS data.A) figures of winged female sphinxes B) fire place, C) decorative stucco frame

Table 2 .
Percentage number of points for specified deviationsfireplace