THE EXAMPLE OF USING THE XIAOMI CAMERAS IN INVENTORY OF MONUMENTAL OBJECTS-FIRST RESULTS

At present, digital documentation recorded in the form of raster or vector files is the obligatory way of inventorying historical objects. Today, photogrammetry is becoming more and more popular and is becoming the standard of documentation in many projects involving the recording of all possible spatial data on landscape, architecture, or even single objects. Low-cost sensors allow for the creation of reliable and accurate three-dimensional models of investigated objects. This paper presents the results of a comparison between the outcomes obtained when using three sources of image: low-cost Xiaomi cameras, a full-frame camera (Canon 5D Mark II) and middle-frame camera (Hasselblad-Hd4). In order to check how the results obtained from the two sensors differ the following parameters were analysed: the accuracy of the orientation of the ground level photos on the control and check points, the distribution of appointed distortion in the self-calibration process, the flatness of the walls, the discrepancies between point clouds from the low-cost cameras and references data. The results presented below are a result of co-operation of researchers from three institutions: the Systems Research Institute PAS, The Department of Geodesy and Cartography at the Warsaw University of Technology and the National Museum in Warsaw.


INTRODUCTION
Generating precise and high resolution documentation of architecture, based on terrestrial laser scanning or dense point clouds from digital images is very challenging and is currently the subject of many studies.In the light of the recent development in the field of low-cost photography and photo acquisition tools we can see the growing potential of close range photogrammetry.Devices such as mobile phones, digital compact cameras or sports cameras are now becoming dependable sources of data, which allow the creation of reliable and accurate three-dimensional models of scenes or objects of interest.
This paper presents the results of a comparison between the outcomes obtained when using three sources of image: low-cost Xiaomi cameras, a full-frame camera (Canon 5D Mark II) and middle-frame camera (Hasselblad-Hd4).The obtained sets of images were used to generate dense point cloud in Agisoft PhotoScan software.A comparison was conducted using data from the same object using full-frame and middle-frame format cameras, which were taken as a reference data of the threedimensional shape.Next, these were compared to the data obtained from the low-cost Xiaomi cameras.The above paper presents the results of the latest investigations carried out by the interdisciplinary team composed of researchers from three institutions: the Systems Research Institute PAS, The Department of Geodesy and Cartography at the Warsaw University of Technology and the National Museum in Warsaw.The subject of the investigation is the frescoes in The Bieliński Palace in Otwock Wielki, a great example of late baroque noble residence.The Palace is located 30 km south of Warsaw.
It is located on an artificial island in the oxbow lake of Vistula.The main part of palace was built in the 80's of the XVII century, and later in the 40's of XVIII wings were added.The form of the palace as well as the rich decoration of the facade is attributed to Tylman van Gameren.

METHODOLOGY
Today, photogrammetry is becoming more and more popular and is becoming the standard of documentation in many projects involving the recording of all possible spatial data on landscape, architecture, or even single objects.
Nowadays, the modern measuring technologies allow to develop complete photogrammetric documentation using the unordered set of images.As a tool for it is appreciated for being easy and fast, requiring little additional work and tools when compared to traditional methods, especially drawing.Since the generated documentation, in the form of images, is orthorectified, any necessary measurements not recorded previously can be taken directly from the documentation.This eliminates the need to refer to the original object for such additional measurements, which might be difficult, costly (e.g.in case of objects documented abroad) or even impossible.The accuracy of the final photogrammetry depends heavily on the amount of data (photos) gathered.While quality of the data is a factor in the accuracy of the final documentation, as it will be shown below, low-cost acquisition tools are sufficient to produce satisfactory results.Kersten and Lindstaedt (2012) show how the archaeological finds and objects can be automatically constructed using low-cost systems.In addition Remondino et al. (2012) analyses some of the low-cost commercial and open-source packages that can automatically process images blocks and find an unknown camera.And there are also attempts to create more complex low-cost systems for the indoor mapping such as described by Kersten el al. (2016).The data gathered for photogrammetric documentation, most notably the point cloud, can be used itself as the entry data for detailed analysis.Such analysis includes for example angles measurement or comparison, or surface flatness.This in turn allows the investigation of geometrical properties of structures.
Today, close-range photogrammetry (image-based) is based on SfM ('Structure form Motion') and MVS ('Multi-view-stereo') approach which are a combination of photogrammetric and computer vision methods.The SfM approach is a fully automated 3D reconstruction technique, which allows to estimate camera orientation (exterior orientation parameters) and self-calibration (Baptista, 2013;Mousa, 2014;Remondino et al., 2014).Image-based approach of an object reconstruction is generally establishing correspondence between primitives extracted from minimum two, but usually more images.Those corresponding primitives are converted into the 3D point cloud using the mathematical model.The main aspect connected with the image-based method is the possibility of the image correlation which is the most important issue during the 3D reconstruction.This problem is studied since more than 30 years, but still many problems exist such as: no possibility of completeness automation, occlusions, poor or un-textured areas, repetitive structures, etc. (Remondino 2008).
The introduction of inexpensive cameras with less stable lenses made it necessary to perform calibration every time an object registrated, due to the possible time-related changes of the camera's parameters.Since the research works on camera field calibration performed by Heller and Brown and Brown's implementation of the independent rays bundle adjustment in 1968, there have been several possibilities for field calibration of close-range cameras (Clarke and Fryer, 1998).These methods are mainly based on taking pictures of test fields and their classification depends on test types, the geometrical conditions of image registration, methods of calculations and the required accuracy.There are several criteria for classification.In order to include the collinearity condition in the calibration process, two basic functional models -the perspective and the projective model -are used (Remondino and Fraser, 2006).
Nowadays, calibration measurements are used in two ways.On the one hand, they are a separate step of calculations, preparing data for adjustment of observations performed during the proper experiment.This procedure was used mainly in earlier closerange projects where measurement cameras had to guarantee the repeatability of geometrical parameters; therefore here we pay attention to metric cameras.The second based on selfcalibration which is implemented in SfM approach.
The term "self-calibration" is understood as calculation of the interior orientation parameters, which takes place during the adjustment process including the calculation of the object point co-ordinates and external camera orientation parameters.This method is the most effective for digital images (Kraus, 1997;Clarke and Fryer, 1998;Cardenal et al., 2004); however, it requires a set of images of different geometry.The differences in calibration models was widely described by the following authors: Brown (1971), Zhang and Yao (2008), Wang (2012) and Markiewicz (2016).

Test field description
The general aim of the project is to document the palace as well as its surroundings.The goal of the completed work, presented, here was to document baroque frescos located in the SE wing of the palace, on the first floor.The frescos cover entirely the walls of a room of 11,5 × 5,5 m.The decoration consists of 9 figural panels limited by a simple frame and Latin sentences below each panel.Preliminary investigation showed that the sketches for the figural panels were made in wet fresco (Fig. 1).The exact technique in which the frescoes were made remains to be established.

Geodetic measurements
A series of geodetic measurements was carried out in the investigated room as well as an adjacent room, in order to establish the ground control points, used for the close-range images orientation (Fig. 2).In order to determine the ground control points the Leica TCRP 1202 total station (Fig. 2b) was used, which allows to measure angle with 2" and reflectors distance measurement with 2 mm+2 ppm accuracy.As a result of the observation adjustment the accuracy of 1,5 mm for control and check points was obtained.Figure 2a and 2c shows the correct distribution of control and check points.

The description of cameras
In the performed experiment, four different cameras were used (Tab.1).
Table 1.The description of the four cameras used The Xiaomi cameras are low-cost devices.The model we used, Xiaomi Yi, has the resolution of 4608 × 3456 pix, the calibrated focal length value of 2,73 mm and the pixel size is 1.34 µm.The other model used, Xiaomi Yi 4K has a lower sensor resolution (4000×3000 pix), the focal-length value of 2,68 mm and the pixel size of 1,55 µm.Both cameras have a wide viewing angle, which will result in a large influence of the radial distortion on the edges of the photos (so-called "fish-eye effect").For the purpose of our experiment both cameras have been mounted on a horizontal bar in order to get a stereoscopic type camera.Moreover external light sources were added (Fig. 3) Figure 3.The distribution of control and check points used in images orientation

RESULTS
In order to verify the possibility of using low-cost Xiaomi cameras as an image source, the following parameters were analysed: • the interior orientation parameters analysis, • the distribution of appointed distortion in the selfcalibration process, • the accuracy of the orientation of the ground level photos on the control and check points, • the total orientation error, • the point cloud density analysis, • the flatness of the walls, • the discrepancies between point clouds from the lowcost cameras and references data.

Interior orientation parameters analysis
On the first stage parameters of the inner orientation were analysed.Fig. 5 shows distortion parameter of Xiaomi cameras.The results of the self-calibration shows that the built-in algorithm for image undistortion, correctly reduce the influence of "fish-eye effect".Additionally the parameters of distortion distribution occurring during the self-calibration process were also analysed (Fig. 6).Before approaching the orientation of the photos we have decided to eliminate the effect of distortion of the lenses through the application of a pre-made algorithm for automatic The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W8, 2017 5th International Workshop LowCost 3D -Sensors, Algorithms, Applications, 28-29 November 2017, Hamburg, Germany distortion removal.The analysis of Figure 6 shows that in the process of self-calibration the parameters of radial distortion have been properly identified.Radial distortion effect is the strongest on the edges of the photos, which proves the correctness of the parameters obtained.This in turn is the prove that this kind of low-cost devices are in fact appropriate for the documentation of architecture.

Image orientation analysis
In order to provide the image orientation acquired from the sensors, the SfM (Structure from Motion) approach has been used.The points from the geodetical measurements were used for orientation of the photos.The accuracy value is never greater than 4 pixels on control points.For check points the value does not exceed 2 pixels, for the Yi 4k camera, and 3 pixels for the Yi 2k camera.These values result from the quality of the images used and location of the points on those photos.When using the photos from the Yi 2k camera, it is impossible to use the points located on the edges of the photos as blurring may occur sometimes.In the case of the Yi 4k camera the impact of this error is smaller, which translates to a higher accuracy of orientation.

Point density analysis
One of the most important factor, which influence on point cloud quality is its density.In order to analyse this factor the Cloud Compare software were used.Figure 7 shows the point density for each sensor is different.It might be noticed that: • the highest point density was achieved for Hasselblad point cloud and the lowest for Xiaomi Cameras, • similar point density were obtained for Xiaomi cameras, • for Xiaomi cameras the point density distribution is regular for the whole area, • comparing low-cost sensor with professional cameras the point density is about 100 times lower, • the results for Yi 4K are better than Yi 2k, • the area without data, when compared to the point clouds generated based on the data from professional cameras, is visible.

Plane flatness analysis -SfM
The second parameter that was verified was the flatness of the point clouds representing the walls.To that aim an app has been created in MatLab allowing the automatic fitting of a plane.In order to achieve that, the function pcfitplane was used.The plane is described by a planeModel object and maxDistance is a maximum allowed distance from an inlier point to the plane.Plane parameters, specified as a 1-by-4 vector.This input specifies the parameters property.The four parameters [a, b, c, d] describe the equation for a plane: ax+by+cz+d=0.
This function uses the M-estimator SAmple Consensus (MSAC) algorithm to find the plane.Figure 8 shows the accuracy of the plane fitting into the SfM point cloud generated from the images taken with the low-cost and professional cameras.The distribution of errors, based on histograms of the deviations from the plane, was analysed.An example is presented in Figure 9.It might be noticed that the deviation for Xiaomi 2k SfM point cloud is lower than +/-50 mm (Fig. 8a) and the histogram of the acquired values is similar to the Gaussian (normal) distribution (Fig. 9a).About 95% of the points do not exceed the deviation of 25 mm.In green are marked the points that do not exceed the deviation of +/-10 mm.For Xiaomi 4k (Fig. 8b, 9b) the quality of SfM point cloud was higher than Xiaomi 2k.The histogram of point deviation from the plane, better approximates the shape of the Gaussian distribution and about 95% of point does not exceed the deviation of +/-20 mm.When analysing the number of points, we notice that more tie points were detected on middle and full frame cameras (Fig 8c,  d).However, the distribution of the tie points is similar and might be treated as a correct.

Plane flatness analysis -MVS
In order to check the correctness of the detected points, the accuracy of the MVS point clouds was performed.Analysing the Figure 10, it might be noticed that the deviation (for all of the point clouds) is lower than +/-35 mm.In red the error higher than 10 mm is marked, in blue lower than -10 mm.The obtained results confirm the expected accuracy of the fitting plane.Based on the presented map of deviation distribution (Fig. 10c, d), it may be stated that the obtained accuracy was higher for point cloud based on Canon and Hasselblad images.
As seen in the diagram (Fig 11c,d), the tested area should not be exactly approximated by the plane (Fig 10 c, d).
When comparing the quality of the 2k MVS point cloud to another dense point cloud, we seem that in the whole area the accuracy is the lowest (Fig. 10a).On the other hand the deviations of the right part, near the door are similar to the Canon reference data (Fig 10 d).The histogram of the devotions from the reference plane does not approximate the Gaussian distribution (Fig. 11a).It's mean that the using the Xiomi Yi 2k does not allow to achieve the high quality of the data.Figure 10b shows that the obtained results are similar to deviations to the plane based on Canon point cloud.

Distance between point clouds
In order to independently check the point cloud's quality, the CloudCompare software was used to analyse differences between fragments of point clouds (so-called octrees).Where products are generated based on the processed point cloud, i.e., orthoimages or 3D models, the accuracy on point cloud is important.Figure 12 and 13 present maps of deviations between Hasselblad and Canon point clouds.For this purpose two independent walls were chosen.The detailed results are presented in Table 3.Unfortunately, the time of processing the Hasselblad middleframe images was about 10 times longer, and authors decided to use only the Canon pint cloud as a reference.In order to examinate the quality of reference data, the distance between point cloud was computed (Fig. 12).It was assumed during calculations that the distance of points is lower than 5 mm.
Figures 13 show that the distribution of differences between the two point clouds are only similar in the in the top part of the wall.During the analysing the distribution of the deviation between the point cloud, it might be shown that the better results was achieved for Xiaomi Yi 4k camera.The Table 3 shows that the mean distance between the clouds generated from the Canon 5D Mark II camera and the Xiaomi Yi 2k and Yi 4k cameras are 4 and 6 mm respectively for 2 wall 10 mm and 6 mm for 4 wall.Comparing the value of the 3σ, the lowest values were obtained for Xiaomi 4k cameras.Given the fact that both Xiaomi cameras are low-cost devices, intended for amateur sport use, we conclude that the presented values are satisfying and acceptable in the documentation of architectural objects.

FINAL REMARKS AND CONCLUSIONS
This paper presents the results of a comparison between the outcomes obtained when using three sources of image: lowcost Xiaomi cameras, a full-frame camera (Canon 5D Mark II) and middle-frame camera (Hasselblad-Hd50).The aim of this project was to analyse the possibility of using the popular low-cost cameras in such purposes.In the course of the investigation the geometrical quality of the results was analysed as well as the radiometric quality.
The following conclusions may be drawn based on the performed experiments: • The quality of the built-in algorithm for distortion correction allows to reduce the sufficient influence of this error, however, it is required to performed detailed analysis for increasing the accuracy.• The disadvantage of using the low-cost Xiaomi cameras is the quality of images which were taken in poor light conditions.This factor not only influences on the correctness of the colour but also on the sharpness of the images.It can be noticed that on the edges of the images the influence of the sharpness is sufficient.• Due to the quality of the Xioami cameras, the ground control points had to be determined on unambiguous defined areas such as corners or contrast parts.• The number and distribution of the tie points detected on images from the low-cost Xiaomi sensor is comparable with the professional cameras.However, the number and quality of these points, which was obtained for Xiaomi Yi 4k device, are much higher than Yi 2k.• The analysis of dense point cloud shows similar quality as a reference data.For inventory and generating architectural documentation, authors recommend the Xiaomi Yi 4k camera.
To summarize, the proposed low-cost tools produced satisfying results in most cases.For the purpose of gaining a rough outlook, the accuracy of the described products is more than acceptable.However, in cases where an exact and detailed registration of architectural monuments is important, using the cameras discussed might be an insufficient solution.
In future work it is planned to develop a low-cost system based on Xiaomi cameras and to improve the quality of photogrammetric products.

Figure 1 .
Figure 1.The example of frescoes which cover entirely the walls of a room

Figure 12 .
Figure 12.Comparison of the points cloud difference between Canon and Hasselblad

Table 2
Table 2 shows the results of the orientation of photos from different sensors.
. The image bundle adjustment accuracy on control and check points according to sensors The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W8, 2017 5th International Workshop LowCost 3D -Sensors, Algorithms, Applications, 28-29 November 2017, Hamburg, Germany The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W8, 2017 5th International Workshop LowCost 3D -Sensors, Algorithms, Applications, 28-29 November 2017, Hamburg, Germany

Table 3 .
Statistics of deviations between point clouds