ANALYSIS OF DATA JOINING FROM DIFFERENT INSTRUMENTS FOR OBJECT MODELLING

The aim of this contribution is an analysis of related technologies for data capturing for creating a digital twin of selected construction. As a case project a baroque church was selected. Four technologies were used for object modelling: TLS (terrestrial laser scanning), digital close-range photogrammetry SfM (structure from motion) as a terrestrial and aerial (drone) technology, and PLS (personal mobile laser scanner). It was possible to combine data from drone and terrestrial photogrammetry, the model was compared to TLS and proved to be sufficiently accurate. The data from the PLS were more complex (inside and outside) but did not contain higher parts of the building. However, it suitably complemented the entire interior. The complex model can be made as joined model from all devices and it is suitable for export to virtual reality (VR).


Documentation methods
Fifty years ago, the documentation of the construction was usually carried out geodetically, often only with a measuring tape. The amount of data was negligible, most of the documentation was in paper form. Nowadays, the technology of the digital twin is a worldwide technology that is strongly developing. The development is possible thanks to new advanced technologies of data capturing, sufficient capacity of computer technology and transmission and information networks (Luhmann et al, 2006).
The current state of knowledge can be characterized as an enormous increase in data about our surroundings due to the rapid development of electronics and digitization. However, the problem is to use this data efficiently and share it with different users online in the current state and with sufficient details. Many theoretical and practical workplaces are now working on solutions that would make the so-called big data accessible to a wide range of users to bring benefits both economically and safety and life cycle management of buildings (BIM, Building Information Modelling); (Remondino et al, 2011, Poloprutský, 2019. The amount of data we can produce with today's technology is enormous, contributed to by remote sensing (RS), TLS, PLS and aerial laser scanning (ALS), terrestrial or "drone" closerange digital photogrammetry (using SfM technology) and several communication networks, which, when used properly, also become a means of navigation. It is essential for further development to solve the problem of storing large amount of data. This is currently being addressed by high-capacity server storage, or other suitable methods.
This case project is focused on integration, analyses and displaying of large datasets made by the latest modern technologies of data collections outdoor and indoor transportation infrastructure. As a case project a historical monument -an unmaintained baroque church-was selected.
Today's latest technologies allow data collection within a few centimetres' accuracy outside and inside infrastructure by various methods of MLS, photogrammetrical SfM and TLS (Thomson et al, 2013), Haala,et al, 2006). Within those methods there are combined precise high-performance laser-scans, GNSS units, IMU (inertial measurements unit) devices, SLAM (Simultaneous localization and mapping) technologies indoor environment, and 360 degrees panoramic cameras (Jaakkola et al, 2008). Today's challenge is to integrate all those inputs such as peta-bytes of coloured point clouds, 3D textured meshes, 360 degrees panoramic images, 3D vector data in BIM. To have a wide range of users, technologies for displaying should be based on modern technologies like WebGL, VR (virtual reality) and optimization on server sites (Pavelka,jr, Raeva, 2019).

Documented object
This case project is focused on integration, analyses and displaying of large datasets made by the latest modern technologies of data collections outdoor and indoor transportation infrastructure. As a case project a historical monument -an unmaintained baroque church from beginning of 18 century -was selected. As a case project the Church of All Saints in Heřmánkovice was chosen. The project of today's church was probably created in 1720 by the famous Christopher Dientzenhofer, or perhaps together with his son Kilian Ignac Dientzenhofer, who led the construction works. Construction began in 1722, the rough construction was built in one year only, however the church was gradually completed until 1726.

USED MESUREMENT OVERVIEW
Based on our long-term project about technologies for 3D documentation of cultural heritage, three easy to use technologies were tested from perspective of time-consuming, financial demands, accuracy, time of evaluation of results and necessary knowledge of the operator, data processing time, and data amount. Depending on transportation problems with large instruments to a distant country, easy to use and easy to transport instruments were used (Matoušková et al, 2021). It means: a) terrestrial close-range photogrammetry, which uses onsite a better digital camera and measuring tape only, b) aerial "drone" photogrammetry using mini-RPAS (remote piloted aircraft system), c) TLS (terrestrial laser scanner), c) PLS (personal mobile laser scanner).
What do these technologies have in common? All create a cloud of points, which is further processed by specialized software.

Terrestrial close-range photogrammetry
After the year 2000, digital cameras as well as the performance and availability of personal computers expanded significantly. This brought the development of software for automatic photogrammetry based on image correlation (SfM). It is necessary to take images with large overlaps from different positions (Hůlková et al, 20109, Patrucco et al, 2019). By this technology, there is a problem of hardly accessible object parts as roofs, as well as possible nearby vegetation and the created point cloud is often very noisy. In our case project, the church was photographed from different positions around the object (parameters in Tab.I). Recently, drones with remote control (RPAS) have become very widespread. This is a modern and rapidly evolving technology for low-cost aerial data collection from smaller areas. Multicopters can be also used for monument care, especially for documentation or monitoring the condition of hard-to-reach parts. You can manoeuvre close to the subject and take both oblique and vertical images. However, the problem is often battery life, camera quality and especially official permits. In this project, DJI Mavic Pro was used, a miniaturized and easy to transport drone. No flight plan was used; the drone was navigated by operator remotely (parameters in Tab.I). The advantage was taking images of upper parts of the building that are not accessible from the surface (the roof, the upper part of the church tower, etc.). Data processing is similar to terrestrial photogrammetry, SfM technology is used, which generates point cloud. Next advantage is that the drone comprises a GNSS device and IMU (inertial measurement unit) which adds to all images necessary information about elements of external orientation; due to the GNSS, the generated model has a scale. Of course, it depends on both IMU and GNSS parts (low cost or precise like RTK GNSS) which give us absolute coordinates with a precision on 1-5cm (like DJI Phantom 4 RTK or Mavic Pro) ( Fig.1-3).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France Figure 3. Textured mesh from joined terrestrial and UAV photogrammetry

TLS
TLS technology has been used since the 1990s. There are many totally different instruments with various accuracy, range, and of course, with different dimension and weight or price. In this project we used the easy to transport miniaturised laser scanner BLK360 (Fig.4). It is measured by the stop and go principle (Fig.5). Data processing is fully automated based on correlation; it works well by simple objects. By complicated objects, there is necessary an operator's action. After this, we got a nice textured model, which can be further processed by specialized software like Geomagic Wrap. A texture transfer between both software systems makes some problems, but it can be successfully solved.

PLS
Mobile personal laser scanners have become excellent helpers in 3D documentation of objects, especially the irregular ones. It is used for its own quick documentation, but also for data conversion to BIM and creation of vector plans. Nowadays, there are more PLS´s on the market, with different resolution, data amount, gear, and price (Leica Pegasus backpack, GreenValley, BLK2GO, ZEB-REVO); (Wang et al, 20109).
The ZEB -REVO personal mobile laser scanner was used. It is easy to use and light to carry. In this version, the camera is not included in the system -it means, the produced point cloud is not coloured from images, only by height like hypsometry. The measurement with the ZEB-REVO is easy -the object must be slowly bypassed from the outside as well as from the inside (Fig.6-9). A problem occurred while scanning. When passing through the narrow tower, despite the slow movements of the operator, the installed IMU unit does not have sufficient accuracy and the model was bifurcated (see Fig.7). After modifying the standard processing parameters in the Geoslam Hub software, the model was processed without errors. We used fallowed new parameters in software settings: under Local: -increase window size to +1, -decrease voxel density to -1, -increase rigidity to +2, under Global: -Increase convergence threshold to +5. For example, a new GreenValley PLS produces really big-data; ten minutes scanning get 2 GB point data and 8GB video-data (Fig. 10). Data from ZEB-REVO are significantly smaller (see Tab.I).

Results
Finally, a joined 3D model was created from both aerial and terrestrial images. It combines benefits from both technologies. At first glance, this seems simple. But even this solution has its problems. First, the same camera is not used, and the internal orientation elements change slightly when shooting automatically.
Next, there can be a problem with excessive differences in the angles of individual photos in ground and aerial images. This means that it is not possible to photograph by the drone only the upper part of the object and the lower part from the surface. The aerial connecting photos from the lower part have to be taken as well, so a data joining can be done.
Data were analysed in CloudCompare software (CC). The results can be well visible in Figs.11-17. Figure 11. Differences between point cloud from TLS BLK360 and ZEB-REVO Go; the reference point cloud was the BLK360 data set (it has better spatial resolution) Figure 12. Differences between point cloud from TLS BLK360 and ZEB-REVO GO Figure 13. Using CC software, the differences are good visible (differences between point cloud from TLS BLK360 and from point cloud made from terrestrial photogrammetry; as the reference point cloud was the BLK360 data set; most of the differences are due to surface roughness textures t Figure 14. Combine terrestrial and aerial (drone) close range photogrammetry model with TLS BLK360 model Figure 15. Histogram-differences between TLS BLK360 and combined photogrammetrical model Figure 16. Differences between point cloud from TLS BLK360 and point cloud generated from UAV photogrammetry; the reference point cloud was the BLK360 data set (it has better spatial resolution). The roofed part of this church was not possible to scan from surface (only very few points were measured using the BLK360 in this part)

¨
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France Figure 17. Histogram-differences between TLS BLK360 and model created using drone The biggest differences can be found by scaffolding and nearby vegetation modelling, which is logical (the accuracy here depends on point dense and vegetation moving). Next, small differences or missing parts can occur on the roof and tower. However, the main parts of measured construction were modelled from all used technologies accurate enough within centimetres (Fig. 7-9).

Accuracy
From the point of view of assessing the accuracy of the outputs, it would be necessary to measure precisely geodetic a set of control points. This was not done in this case project; the only option was to use CloudCompare software and define what is the most accurate in a theory. From the point of view of measurement and possible interfering noise, the TLS method is the most accurate (i.e., BLK360, although it is not an accurate laser scanner). It can be said that after testing the accuracy earlier and based on the description of the device from the company, the accuracy is 4 mm per 10 m. Most distances were about 10 m. The model from TLS was used as a reference and models from drone, terrestrial close-range photogrammetry, PLS, and the combined photogrammetric model, all were tested with the reference model. The complex model from all measurements is always slightly modified distortion due to small differences, but it has cm accuracy. In our case the maximal difference reached 6cm, typical 2-3cm, which should be good enough for most needs.

Visualisation
The final point cloud can be analysed in CC software and next meshed. A meshed model, after decimating and editing, is an input for a game engine like Unreal Engine (UE). From UE can be generated a virtual model for VR (Fig.18-21).

CONCLUSION
The amount of data for modern object documentation is currently in the order of tens to hundreds of GB. Terrestrial close-range photogrammetry and aerial low-altitude photogrammetry from a drone is low-cost, relatively quick but each technology has its disadvantages. Certain parts of the object are always missing. Certainly, many people think that it is ideal to combine the benefits of a drone with terrestrial photogrammetry. Yes, it is possible, but it has also its problems. From one point of view, different cameras are used, and there cannot be a big difference in distance from the object between the two systems. Oblique images must follow each other, i.e., the height angle between ground image axes and aerial images axes cannot be greater than approximately 30 degrees, and otherwise, the images will not combine in one model. It is true that sub-models from different sources can be combined into one later in other software (e.g., Geomagic Wrap), but it is time consuming. After many hours, a joined model from both aerial and terrestrial images was created directly in Metashape, not all images were oriented (14 images from 480 were not possible to orient), but it was not necessary -the resulted 3D model was complete. The BLK360 laser scanner was good for precise 3D modelling of this object; of course, there are missing parts on the roof and data collection and a processing takes several hours.
The personal mobile laser scanner is the simplest way, how to make the 3D model. However, the data amount and the precision from ZEB-REVO are not as high as by TLS, but good enough for a historical construction. Data is obtained very easily, and processing is fully automatic. In addition, in the associated Geoslam Draw module it is possible to vectorise semi-automatically created ground plan orthophoto. This works well for simple objects, but a lot of manual work is required for complex and historical irregular objects. A disadvantage is that a digital camera is missing by the simplest ZEB-REVO Go PLS. With using of more expensive instruments like BLK2GO or GreenValley PLS is the point cloud textured directly from photos. All point clouds are expected to be joined into one model. This is possible, but it turns out that each technology has a different accuracy and a different basic use. Thus, the overall joining gives a highly noisy model with several differences and duplicate parts affected by local small errors. Creating a quality overall model requires a lot of editing by the operator. Due to various requirements, it is therefore suitable for different users to make the original georeferenced data available as individual layers, rather than creating only a single model from different data. Different data are needed for VR, it is necessary to join all measured data to one complex model, which does not have to be maximal precise and does not have contained dozens of millions of object points, different requirements are for the creation of precise construction 2D plans or for architectural studies (Fig.20-21).