POINT CLOUD MANAGEMENT THROUGH THE REALIZATION OF THE INTELLIGENT CLOUD VIEWER SOFTWARE

The paper presents a software dedicated to the elaboration of point clouds, called Intelligent Cloud Viewer (ICV), made in-house by AESEI software (Spin-Off of Politecnico di Bari), allowing to view point cloud of several tens of millions of points, also on of “no” very high performance systems. The elaborations are carried out on the whole point cloud and managed by means of the display only part of it in order to speed up rendering. It is designed for 64-bit Windows and is fully written in C ++ and integrates different specialized modules for computer graphics (Open Inventor by SGI, Silicon Graphics Inc), maths (BLAS, EIGEN), computational geometry (CGAL, Computational Geometry Algorithms Library), registration and advanced algorithms for point clouds (PCL, Point Cloud Library), advanced data structures (BOOST, Basic Object Oriented Supporting Tools), etc. ICV incorporates a number of features such as, for example, cropping, transformation and georeferencing, matching, registration, decimation, sections, distances calculation between clouds, etc. It has been tested on photographic and TLS (Terrestrial Laser Scanner) data, obtaining satisfactory results. The potentialities of the software have been tested by carrying out the photogrammetric survey of the Castel del Monte which was already available in previous laser scanner survey made from the ground by the same authors. For the aerophotogrammetric survey has been adopted a flight height of approximately 1000ft AGL (Above Ground Level) and, overall, have been acquired over 800 photos in just over 15 minutes, with a covering not less than 80%, the planned speed of about 90 knots.


INTRODUCTION
Terrestrial laser scanner (TLS) and photogrammetric techniques are increasingly used in order to modeling both natural and anthropic surfaces in 3D at an unprecedented resolution and precision.The technique TLS (Terrestrial Laser Scanner), widespread since the 2000s, has evolved rapidly in the field of research and development for the acquisition of three-dimensional information about a variety of sites and objects, such as cultural heritage and architectural, coastal cliffs, etc.A TLS survey enables rapid and accurate capture of a dense point cloud of millions of points in a short period of time with a high degree of detail, not achievable with traditional survey techniques.Projective geometry, instead, represents the basis for the developments in photogrammetry in the late 19th century, when Aimé Laussedat experimented with terrestrial imagery as well as kites and balloons for obtaining imagery for topographic mapping (Laussedat, 1899).In the last two decades the Photogrammetry has become an essential tool in the territory survey combined with development of increasingly high-performance digital sensors and the major commercial software innovations and opensource, allowing the generation of increasingly complex 3D models from digital stereoscopic images.In particular, the acquisition of oblique photos from aircraft with pilot, together with the Unmanned Aerial Vehicle (UAV) technology, allows the generation of high quality models at a relatively low cost.The increasing computing capability and the rapid developments in the field of Computer Vision have led to the Structure from Motion (SfM) method, which has opened the way for the high-resolution and low cost topography.The SfM, in fact, can also be applied to digital images acquired with amateur cameras, therefore, it is very flexible in its application (Eltner, 2016;Masiero, 2014).However, the method requires a design phase very accurate of photographic acquisitions, as well as an accurate phase image acquisition to avoid possible errors, shadow zones and to ensure a good quality of the reconstructed scene.The paper presents a software dedicated to the elaboration of point clouds, called Intelligent Cloud Viewer (ICV), that provides the integration of point clouds derived from the TLS and photogrammetric survey (Clapuyt, 2016).As described below, the potentialities of the software have been tested by carrying out the Photogrammetric survey of the Castel del Monte which was already available in previous laser scanner survey made from the ground by the same authors.Widely known for its distinctive octagonal shape and for being the most mysterious among Frederick's buildings, Castel del Monte is one of the main tourist destinations in Puglia.The castle, from a height of 540 meters a.s.l., dominates a wide area of Puglia and Basilicata (fig.1).In 1996 Castel del Monte has joined the UNESCO sites, in fact, the Commission considered it "….asite of outstanding universal value in its formal perfection and the harmonious blending of cultural elements from northern Europe, from the Eastern world and classical antiquity ... Castel del Monte is a unique masterpiece of medieval military architecture, an expression of its founder Frederick II of Swabia…" (fig.2).On 29 January 1240, the emperor Frederick II signed a decree ordering him to prepare the material needed to build of a castle located at the Sancta Maria de Monte church (no longer extant).Some sources report that in 1246 Manfredi, son of Frederick, locked up in the castle a few rebellious subjects, and in 1249 there took place the celebrations for the wedding of Violante, the emperor's natural daughter with Riccardo conte of Caserta.In a navigation manual dating back to 1250, known as Compasso de navigare, is reported "…una montagna longa enfra terra et alta, e la dicta montagna se clama lo Monte de Sancta Maria, et à en quello monte uno castello…", testimony to the fact that the building, which is visible in the coastal stretch between Trani and Barletta, was a point of reference for navigation.Beginning in 1277, Charles I of Angiò strengthened its function of sighting and control of the territory (Licinio, 2002).Except for short periods of time dedicated to the festivities, the Castle was mostly used as a prison.Its current name appeared, subsequently, in a decree of Ferdinand of Aragon.Annexed to the dukedom of Andria, it belonged to Consalvo of Cordova and from 1552, the Carafa, Counts of Ruvo.Moreover, during the plague of 1656 it was a refuge for many noble families of Andria (Mola, 2002).In 1876 the Castle was bought by the Italian State for the paltry sum of £ 25,000 who started the rescue and restoration work.The restoration work was resumed with continuity and scientific caution in 1928, until you get the last interventions of the eighties.The structure of the Castle is a monumental block of octagonal shape at whose eight vertices there are towers with of the same shape (fig.3).The distribution of the internal space is articulated on two floors, each of which has eight trapezoidal rooms arranged around a courtyard which is also of octagonal shape.The main façade, on the east side is dominated by a majestic portal being accessed by two symmetrical flights of stairs.The courtyard, compact and severe, which repeats the octagonal setting throughout, lightens his wall mass.On the inside there are three portals of access to rooms on the ground floor and three doors windows that correspond with other rooms on the upper floor (Schirmer, 2000).

TLS and topographic survey
The survey activity has been designed on the basis of a simplified 3D model (fig.4) obtained from the available plans in the bibliography.In particular, for the topographic survey was adopted a local reference system, used for the next georeferencing of laser scanning data (Capra, 2005, Dardanelli, 2017).
The topographical scheme adopted is based on a polygonal closed instituted at the ground floor; however, the complete closure of other polygonal was not possible for the single floor because the rooms I and VIII, II and III of the ground floor and the rooms I and II of the upper floor are not communicating with each other.The solution to connect the main closed polygonal line joining the vertices outside the building and, therefore, countervailable, to the secondary of the ground floor in the direction of the entrance portal has allowed to stiffen the geometry of the network.
The topographical network of support overall consists of 28 station points, eight of which are outside of the Castle.From each station were detected the points needed to detailed survey.
The processed data with the dedicated software have been exported to .txt and .dxf(fig.5).
In order to ensure coverage of the entire building and to minimize the shadow areas a total of 92 scanword TLS have been acquired (fig.6).The coverage of the entire area has been articulated considering the topographical support points available for each room and the necessary laser acquisitions so as to relate them to one and the same system and to minimize any shadow areas.The registration of scans has been realized by means of the Cyclone software, proceeding first alignment between the different scan for each environment (fig.7) and, subsequently, to the georeferencing of the entire building (fig.8).
The average standard error obtained is in the range 0.008-0.015m, therefore, contained in the requested accuracy of the model.Subsequently, the 3D model surfaces was generated.

Survey with from oblique photogrammetry
The photogrammetric survey has been carried out on board a single-engine Socata TB10, light aircraft of the General Aviation (fig.9).The abundant fenestration and the variable pitch propeller have enabled a wide view and excellent maneuverability and fluidity of piloting also in turns.The availability of a high sensitivity of the sensor, a high resolution of the same, a fast autofocus system (in order to obtain every time images sharpened and in focus), a professional optics (very bright and with low optical aberrations and color) chromatic and optical aberrations), an SD (Secure Digital) storage class 10 ultra (SDXC ultra-fast recording of images even in bursts at full resolution and in RAW format without loss of quality) ensured a secure and comfortable recovery phase, while minimizing the number of unusable photos.
For the whole mission of the aerophotogrammetric survey, that lasted just over 1 hour, has been adopted flight height by approximately 1000 ft AGL (Above Ground Level) with a planned speed of about 90 knots.Overall, were acquired over 800 photos in little more than 15 minutes, considering the acquisition of 1 photo per second, approximately with an overlap not less than 80%.
The photos were acquired with aperture priority mode with values between f11 and f14, thereby ensuring exposure times between 1/1500 and 1/2500 of a second or so, more than enough to avoid the occurrence of the unwanted image blurring.

Generation of the point cloud by oblique photos
The photogrammetric point cloud of Castel del Monte was obtained in Agisoft PhotoScan (fig.12), which provides photogrammetric restitution by mean of image processing and SfM (Structure from Motion) techinques.
The high number of photo used (over 300), the high resolution and quality of the same and the high overlap of the images have made it possible to obtain a very detailed and dense point cloud (over 36 million of points).
Figure 12.Point cloud in Photoscan

ICV (INTELLIGENT CLOUD VIEWER) APPLICATION
Given that the scale of the photogrammetric model obtained was arbitrary, for georeferencing of the point cloud have been used the at the fine registration instruments implemented in the ICV software developed in-house as part of the research activity by the authors.Specifically, analyzing the acronym, intelligent wants to emphasize the possibility to manage, process and visualize complex point cloud, even tens of millions of points, also on no high-performing computers.Therefore, the elaborations are applied and executed the entire point cloud, while the display is handled in an "intelligent" manner, displaying only a part of the cloud in order to speed-up rendering (fig.13).Inventor), numerical calculation (EIGEN, BLAS), computational geometry (CGAL), registration and advanced algorithms for point clouds (PCL, Point Cloud Library) and advanced data structures (STL, BOOST).The adoption of the Object-Oriented paradigm for code reuse and the extensive use of templates ensure an easy extendibility and a substantial reduction in the time of deployment and debugging.The basic structure, in the process of further development and extension, integrates a series of functionalities in the form of tools and filters that allow you to carry out the main operations on point cloud, such as, the color filter, the cropping, the affine transformation to 7 parameters, the matching (coarse registration), the registration (fine registration), the simplification (decimation), the generation of sections and the calculation of distances between clouds.For example, the color filter allows to eliminate outliers which do not belong to the object in addition to the edge effects according to the scheme in figure 14.In particular, it is applicable on the point cloud by manually selecting the single point or a selection box which corresponds to an RGB value or manually typing the RGB value in the dedicated window (fig.15).The photogrammetric model obtained was subsequently simplified applying the tool Simplify (fig.18).This is applicable on the whole point cloud or on one of its selection, defining a step or a percentage of decimation (fig.19).The simplified model was then scaled using the tool Transformation (fig.20) executable using the Helmert transformation disposing of GCPs (Ground Control Points) of which are known to the cartesian coordinates, as in the case under examination, or with respect to a reference point cloud (match) (fig.21).In order to georeferencing the scaled photogrammetric model, ICV makes a fine registration, using an optimized version of the ICP (Iterative Closest Point) (Vassilaki, 2012) family algorithms.The algorithm is capable of automatically align two point clouds, minimizing errors.In particular, the photogrammetric point cloud has been "adapted" on the TLS model of Castel del Monte assumed as a reference cloud (target) by means of an affine transformation (fig.22).In order to obtain continuous 2D representations (planimetric, transverse and longitudinal) from 3D laser model and obtain discrete maps metrically valid, it was adopted an experimental numerical algorithm developed by the authors (Costantino, 2011), originally written in Matlab language and, later, optimized in ICV.In particular, starting from the selection of a slice through the tool Section, manageable both in manual mode (with manipulator box) that by keyboard (fig.23), is possible to set the position of cutting planes, the number, of section planes, the thickness of the slice, the interpolation step, the distance between consecutive sections, as well as to set the orientation of the same (horizontal, vertical, or any oblique attitude) (fig.24).The points in correspondence with the section planes are first "ordered" according to the criterion of mutual distance and then decimated with a constant pitch by means of a linear interpolation.These points, characterized by coordinates (X, Y, Z) are, subsequently, converted into 3D polylines exportable in .dxfformat.The generated output (fig.25) have been exported to dxf file (Costantino, 2013) and, therefore, manageable in a CAD environment (fig.26).

COMPARISON BETWEEN PHOTOGRAMMETRIC MODEL AND LASER SCANNERS MODEL
In order to evaluate the accuracy of the photogrammetric point cloud was used the M3C2 algorithm (Multiscale Model to Model Cloud Comparison) (Lague, 2013), implemented in ICV environment.This allows to evaluate the errors in terms of distances between two point-clouds, in a robust manner because discards any unwanted outlier and effectively by managing the edge effects (Hodge, 2010).By selecting interactively on-screen a portion of the two registered point cloud (fig.27) was possible to evaluate the distance between the two point clouds.The results are displayed in both the terms of the map and the histogram (fig.28).
The comparison is repeatable for any portion of the object detected in order to quantify an average precision of the method.In the present case, has been obtained an average error (in terms of distance) in the lower absolute value than 2 cm, with an RMS error of the order of 2 cm.
Figure 28.Output comparison with M3C2

5.CONCLUSIONS AND FUTURE PROSPECTS
The TLS survey has returned a precise model of Castel del Monte, although has required a great deal of work both in the survey phase (about 6 days) and in elaboration (over 50 days).
The aerial survey of the oblique photo, instead, was performed in about an hour (therefore, with personnel costs and reduced instrumentation) and has enabled through the use of the Structure from Motion to obtain a 3D model of the object valid metrically also if, however, it is refers only to the outside of the site visible from the airplane.The precisions of the photogrammetric model, evaluated in a robust manner with the M3C2 algorithm, in fact, for the architectural representation purpose are acceptable like those obtainable with the TLS survey.
In addition, commercial software currently in use for TLS data processing, they must be highly performance and they do not always respond fully to the needs of the user in the management of information that require high computational performance and of visualization.Therefore, the ICV software, is able to respond in a simplified manner to the needs of the user, appears to be a goal and its applicability on cloud point is independent of the origin of the same.
In addition to the functionality already present in ICV, we want to implement new tools such as, for example: creation of surfaces, orthophotos, robust processing of the acquired images, etc. Particular attention will be devoted to algorithms for the fine registration.ICV, in fact, currently for the merging/alignment of point-cloud coming from different sources (eg.TLS and photogrammetric, or surveys of the same type but carried out independently or at different times) integrates algorithms of ICP type.Besides these algorithms, which have a purely local character, we also want to integrate robust fine-registration algorithm of global type as the 4PCS (4points Congruent Sets) and/or its optimized variants based on smart indexing and key-points extraction criteria.Are being integrating also algorithms for image processing (OpenCV) for automated classification of point clouds starting from the color information of the point and the analysis of the boundary conditions (Brodu, 2012); for the emulation of LiDAR surveys but with purely photogrammetric techniques; for the extraction of solid geometry useful for the of automated restitution of entire urban centers (Fischler, 1981).Also, we want to implement a library of permanent scatters distributed on the regional territory in order to transform and/or georeferencing any object surveyed in flight, further reducing significantly the time and costs to the ground and enhancing the survey by means of oblique photos of individual structures or rather on urban and sub-urban areas or environmental and specific elements of the territory.

Figure
Figure 1.Territorial framework

Figure 7 .
Figure 7. Assembly of the ground floor point clouds

Figure 9 .
Figure 9. Oblique photos acquisitionFor the photographic images a digital SLR Nikon D5500 resolution was used with 24.2 Megapixel and equipped with an ultra-fast autofocus system with dual-mode detection both of contrast and phase.Additionally, it has been used a photographic zoom Nikkor 18-70 mm, by adopting a focal length of 70 mm for all the photos, with ISO values equal to 1600 (purposely kept low compared to the maximum of ISO 25600 available in order to minimize electronic noise).The availability of a high sensitivity of the sensor, a high resolution of the same, a fast autofocus system (in order to obtain every time images sharpened and in focus), a professional optics (very bright and with low optical aberrations

Figure 10 .
Figure 10.Flight Planning in FlightPlanIn order of an easier and more expeditious flight planning has been used the application FlightPlan (fig.10),implemented inhouse.In addition to the paper-based information (aeronautical charts and flight logs) the pilot made also use of an electronic knee pad (fig.11)useful to constantly monitoring the aircraft's position with respect to the object of the survey, in addition to improve the situational awareness and, more of all, fundamental to quickly schedule any diversions in case of aircraft critical failure or in emergency situations.

Figure 13 -
Figure 13 -Viewing point cloud management in ICV ICV is an application developed for 64 bit Windows systems which allows a very large address space, such as to allow the management of point cloud constituted by a virtually unlimited number of points.Written entirely in C++, it integrates different modules implemented by the authors with open-source algorithms specialized in computer graphics (SGI Open

Figure 14 .
Figure 14.Color filter The tool Crop, instead, acting on the single or multiple view (fig.16), allows to cut individual portions of the object and to eliminate portions of the model not of interest (fig.17).

Figure 23 .
Figure 23.Definition of the section plane

Figure 26 .
Figure 26.Sample section in CAD