HOLOGRAPHIC VISUALIZATION AND MANAGEMENT OF BIG POINT CLOUD

: The work aims to present and validate the workflow from the 3D survey to the visualization of cultural heritage objects using the innovative Euclideon Hologram Table©. Three case studies surveyed with three different systems and at three different scales have been selected: Santa Maria delle Grazie in Milan (terrestrial laser scanner for an architectural scale), the village of Ghesc in the Ossola valley (UAV survey for an environmental scale) and the cuneiform clay tablet number 727 (structured light system for a detail scale). The whole process of transforming the 3D point/mesh model to hologram was verified, analysing the file formats, technical performance and specifications, file dimensions manageable, and details viewable. The first test shows great potentiality, because the hologram exploring is impressively fluid even when zooming to view a higher detail level, despite the high number of points/polygons. The power and performance of the point cloud 3D rendering engine result impressive. Nonetheless, different aspects need further research, from point cloud visualization quality to enhancing 3D model interaction.


INTRODUCTION
This paper aims to describe and check the workflow from the digital survey to its implementation in an innovative holographic table of several case studies surveyed with three different 3D measurement systems and at three different scales: territorial (the small village of Ghesc), architectural (the UNESCO heritage monument of the church of "Santa Maria delle Grazie" and the Convent seat of the Cenacolo Vinciano) and detail (a small clay tablet with cuneiform engraving). Innovative representations and graphic solutions useful for the analysis of each of the cases examined will be shown. The research topic concerns the verification of the whole process for the raw survey dataset conversion in a hologram, considering the remarkable potentiality of this visualisation system. It allows fluid navigation in the point cloud, zoom in to view tiny details and a real-time zoom out to view the entire model.

Relevance
The relevant topic regarding transformation from twodimensional to three-dimensional visualization devices has grown exponentially in recent years, driven by the need for applications in industrial and medical fields (Pierdicca 2017;Fraga-Lamas 2018;Brun 2019;Desselle 2020). Virtual Reality combined with hologram visualization is increasingly applications in the field of 3D geo-information too (e.g., to visualizing the holographic representation of energy production methods, Virtanen et al., 2020). On the contrary, the world of Cultural Heritage is lies somewhat behind, with applications related mainly to tourism and education (Blanco-Pons et al., 2019;Voinea et al., 2019). However, in the last years, thanks to holographic devices such as Microsoft HoloLens 2 it is possible to see some initial application oriented to the professional practice and visualization of complex heritage buildings and scenarios Fassi, 2021a, Teruggi andFassi, 2021b). Virtual Conservation (Reaver, K., 2019 studies and applications are still being developed to integrate virtual reality into traditional architectural and heritage preservation practices. * Corresponding author While the techniques of the digital survey have had an impressive progression since the advent of the first laser instruments in recent years, both for accuracy and data resolution, the methods of digital model projection have developed improvements on processors and components but not on the mathematical models underlying the projection, always performed through 2D screens. Until a few years ago, Holograms (from greek "hòlos" everything and "gràmma/graphèin" graphic) were commonly associated with the optical process capable of reproducing 2D images with a certain depth used as static images in devices such as credit cards or marks on banknotes. In their most current version, mediated by the futuristic images of Star Wars, they refer instead to a 3D image that occupies the space in front of us, providing the digital reproduction in 3 dimensions of the projected objects. Objects can be viewed from different angles, and in the most emancipated version, could also develop a type of dynamic holography. The development was made possible by the radical improvement of the mathematical models, and thus of the interferences, called cross-talk, that were created between the large number of two-dimensional images assembled (to create a three-dimensional effect), which ended up generating blurring and nullifying perspective depth (Makey, 2019).

The Compex of Santa Maria delle Grazie
The whole complex of "Santa Maria delle Grazie" (Figure 1) is surveyed in its present state, therefore including the transformations carried out by Guiniforte Solari first, ordered by Ludovico il Moro during the Renaissance period and realized through the works of Bramante and Leonardo, the demolitions and restoration carried out by Luca Beltrami and the reconstructions occurred after the bombings of 1943 (Bruschi, 1983). Due to the large dimension and the complexity of the structure, the convent survey had provided more measurement campaigns. Moreover, the case study for its architectural importance, for its constructive transformations and significant survey problems connected to its shape, was considered an in-situ research laboratory. The original Terrestrial Laser Scanner datasets can be divided into two macro families. A first laser scanning survey campaign was conducted in 2019, using a Leica HDS7000 for the "cloister of frogs" (16 scans) and a Leica P30 for the cloister of the prior, the old and the new sacristy (36 scans) (Bolognesi and Fiorillo, 2019). The merge of both TLS survey reaches more than 2 bln points. The second survey campaign more recent (2020-21) was set up with a more innovative instrument, Leica RTC 360, to scan the dome (interior and exterior surface) and outer perimeter of the church on other two days of in situ activities. This last 3D survey implies 122 scans reaching around 4 bln points. Each TLS survey campaigns have generated a related project with all scans registered in a local coordinate system. All projects are now managed and merged using the recent software environment of the Leica Cyclone REGISTER 360, which allows the cloud-to-cloud registration to align: i) the new scans acquisitions; ii) all 'old' scan group in the same reference system. A rigid roto-translation is applied to the scan blocks of each survey project. Each of the 174 aligned scans was sampled to 5mm resolution, removing duplicate points to obtain the global point model of Santa Maria complex. This is the most interesting case study for the research presented for two main reasons. First, it took many scans to 3D measures the indoor and outdoor intricate spaces with a high resolution (millimetric order). Therefore, the workflow from raw data to the hologram is multiple steps and time-consuming. Furthermore, the related point model has the lowest point-to-point spacing and the higher number of points, and therefore it is more complex to manage and visualise.

The Ghesc village
The small medieval village of Ghesc is hidden in the Ossola Valley (Montecrestese -Italy) among the Piedmont mountains and is formed by nine stone houses (Associazione Canova, 2010). A photogrammetric UAV survey of the rural site was conducted during the annual "Summer School: Surveying Cultural Heritage -Laboratory of Places" organized by the 3DSurvey Group -Politecnico di Milano (Achille et al., 2017). The 3D model of the central group of 5 building is composed of about 18 million points (119 photos by Phantom 4 DJI and an average GSD of 6 mm) and interested an area of about 500 mq ( Figure 1).

The 727-cuneiform tablet
The clay tablet number 727 with cuneiform 1 engraving was 3D measured with a structured light scanner (scan in a BOX @2015 Open Technologies SRL) by 3D Survey Group (prof. Francesco Fassi and Fausta Fiorillo -Politecnico di Milano) as part of a collaboration with Prof. Catherine Mittermayer (Department of Antiquity Sciences, University of Geneva) and the Assyriologist Mirko Surdi (PhD FWO Fellow at Ghent University). The interdisciplinary research aims to obtain a digital clone of the cuneiform tablets with high details and high accuracy for education and scientific purposes (for example, share studies and discoveries among archaeologists everywhere to decipher the text content). The physical dimension of the tablet is about 3x5x7cm. The engravings are tiny, some of the size under half a millimetre, and the characteristic wedge-shaped footprints are 1-5mm deep. The final digital model is formed by 2.586.462 polygons with a minimum resolution of 0.078mm ( Figure 1). This case study is very interesting to verify how the device manages the holographic visualisation of even small objects.

Hardware setup
The Hologram table ( Figure 2) (one of the few examples in Europe for research) is one of the equipment of the new LaborA -physical and virtual modelling laboratory (Politecnico di Milano). It is composed of a large and flat surface (2.1m x 2.1m) and a metallic frame structure to which are attached all technological components required for the holographic display of 3D models as well as the wooden panel of the top plane. The brain of the device is a standardised Dell 5820 workstation (Dell, 2020) equipped with two Radeon WX 5100 graphic cards (AMD, 2020). One oversees rendering the view to a standard external projector. The other is used by the four main projectors of the hologram table to display the 3D model on the flat screen. The four projectors are Vivitek D757WT (Vivitek ,2020) with the maximum supported resolution of 1920 x 1200 at 60 Hz, modified adding plastic support for Euclideon filtering crystal. The latter has the task to filter light rays, which are then recomposed in the final holographic 3D model. An eight-port ethernet switch allows to connect through a local network the projectors and the PC to a Crestron remote controller commanding the hardware power on/off and the start/stop of the software for holographic display. Interaction with the holograms is performed with a specially designed wand, which, as though the glasses, uses an infrared tracking system to calculate their position and orientation in space. Figure 3 shows the wand, glasses and one of the four tracking domes. The spheres on the wand and the glasses are used to compute their correct position. Tracking equipment consists of tracking domes and controller, Radio Frequency Dongle and Sync Emitter, which are USB connected to the PC. Sync emitter will receive light from glasses and wands and sync all devices to work together. Tracking domes work on the infrared wavelength of light, allowing tracking wand and glasses position through the working session. The flat-screen area comprises a rear-projection material (white cloth) that is sandwiched through a 10 mm thick acrylic sheet and a 1.5 mm one. This area (1390 mm 2 ) is the central display where holographic display takes place. The system projects the 3D models so that they appear to rise from the centre of the table to a height of approximately 0.7m in a hemispherical volume. The number of direct users (equipped with wand and glasses) that can view and interact with the hologram is two. for each primary user, a secondary user can be added that must be on the same side of the table as the primary user. It is an innovative Holographics Multi-User Technology; usually, multiple people cannot view holograms simultaneously.
Proper functioning of the hologram table requires a completely dark room, working with projectors and lasers. Natural and artificial light obviously interfere with the display of holograms and with the tracking devices.  Furthermore, when preparing the environment for the hologram set, it should be considered that there should be no infrared devices that work with IR light (including some alarm systems with motion detection) and mirrors (within 4 metres).

Software setup and 3D digital model requirements
Euclideon Hologram Table© manages 3D models using specific software for data preparation and display on the flat table screen: i) Holoverse Present ii) Holoverse Professional and iii) HoloTray (Table 1).   (Euclideon, 2013) is a 3D graphics engine based on points search indexing system. The engine uses the search algorithm to define which points of the 3D model must be displayed. The algorithm records only one point for each pixel of the screen. Therefore, the disk space required to store the point cloud data define the geometric detail level, and the screen resolution determines the rendering speed. This technology provides excellent graphics power and allows the efficient rendering of hypothetically unlimited amounts of points, offering detailed visualization in real-time.
All the digital survey data elaborations necessary for the holographic visualization and presented in the current paper were performed on the same desktop since both the conversion times and the maximum file size that can be processed strictly depend on the computer hardware available Table 2). In this way, we make sure that the data provided by the tests carried out are standardized concerning the potential and technical specifications always of the same computer. The Holoverse Present imports point cloud models only in LAS file format, while it is less restrictive for standard 3D mesh models format (Table 3). However, different file extension can be associated with the generated point/mesh model depending on the type of survey technique used. Therefore, the data coming from the survey campaign must be uniformed to certain criteria; therefore, their coherent and systematic management is essential. Figure 4 shows the main steps of the workflow from the survey to the holographic display.

3D Mesh
OBJ PM PLY FBX DAE DXF STL Table 3. Holoverse Present supported file formats.
When dealing with terrestrial laser scanner surveys, in general, it is not possible, and not even convenient, to export directly from the instrument proprietary software a cloud-ready for holographic visualization in LAS format. It is often advantageous to export the final product of the elaborations in the E57 file extension for three main reasons. The first is because this opensource file format allows the storage of the 3D point data and different layers of information, that is the attributes such as intensity, surface Normals, RGB colour, and associated images (Huber 2011). The second reason is that for the same number of points in the cloud, this format compresses the data so that the file size is smaller than that of the equivalent file encoded in ASCII format (e.g., PTX or TXT). It used, instead, binary storage and data compression. Even reading and especially writing in E57 format should be faster than the equivalent file in ASCII format. Lastly, it is also a format compatible with many point clouds management software. For these reasons, this file format was chosen and used within the tested workflow related to the laser scanning acquisition of the Santa Maria delle Grazie complex. Photogrammetric point clouds can be exported directly to different types of file formats, encoded as ASCII or binary. Therefore, almost always, the photogrammetric software allows saving also directly in LAS. However, it is often necessary to edit the point cloud (cleaning and sampling) in more performing external software. The product of the triangulation-based scanners is often directly a polygonal model (STL, OBJ, PLY). In fact, it is also possible to export the point cloud of the digital model but given the small size of the surveyed objects in general, they are optimized well in the mesh format. However, independently from the original file extension of the 3D survey, it is possible to process the final products Cloud Compare (CloudCompare, 2020) environment, a GPL (General Public License) software. Among numerous functions, this software allows both to clean all remained unwanted points from the 3D point model and subsample it at the desired resolution. Furthermore, it is possible to manage which type of information/attributes will be exported along with the final point cloud, encoded in binary LAS format for Holoverse Present.
To prepare point cloud models for the holographic visualization, it is advantageous to know some characteristics (e.g., the point data record format) and proprieties of the LAS file (ASPRS, 2019). Two aspects are essential for this research: i) X, Y, Z coordinates are stored as "long integers"; ii) Red, Green and Blue data are of type "unsigned short" (with values that range from 0 to 255 for each colour channel). Even if the LAS format allows storage of additional fields as Intensity, Return Numbers, Scan Direction Flag, Classification, etc., Holoverse Present will skip all values after the colour information. Similarly, the information on Normals to the point The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China that usually helps the display the non-coloured point cloud is not supported. In the case that RGB colour is not provided, Present software assigns a uniform colour to the point cloud, loosing display quality and depth (e.g., black model on the right of Figure  5). Therefore, if the RGB values were not acquired during the laser scanner survey campaign, it is mandatory to convert intensity field of the scans to their equivalent RGB information.
(a) (b) Figure 5. a) Point cloud with Normals displayed inside CloudCompare. b) The same point cloud displayed in Holoverse Present.
After exporting the LAS file from CloudCompare software (or the mesh file from property scanner software) a further step is necessary. The point cloud model must be imported in Holoverse Present software so that it can be converted in UDS file format and a UDP presentation can be prepared. Although it is not essential, this presentation, allows saving different predefined view of the 3D model that speed up the navigation and presentation process of the holographic model. Table device is presented to the user to adjust the visualization of the digital model according to the physical size of the holographic screen. It is possible to move, scale, and orient the displayed model related to the device in the virtual environment. The simulation allows defining which part of the model will fall inside the dome rendering area above the table, thus being correctly displayed, and which part may possibly remain outside, yet resulting in invisible to the user.

Inside the Holoverse Present software, a virtual view of the Hologram
Once 3D model positioning is performed, it is possible to convert it to UDS file format and to save it locally to disk. This export has two main advantages. The first is that it converts the point model in a format optimized to be managed by Euclideon software, increasing rendering capabilities and overall smoothness of the experience. The second is that file size is incredibly decreased, resulting in more effortless transferring and handling models (Table 4).  Table 4. 3D model management pipeline data.

METHODOLOGY
The three test cases have been selected for their different peculiar characteristics in dimension, different type of survey, accuracy.

The Santa Maria delle Grazie complex
The final cloud of the Santa Maria complex is composed of 174 scans registered in the same local coordinate system. The single point clouds have been exported from Leica Cyclone REGISTER 360 in E57 file format, allowing manageable subsequent processing inside the CloudCompare environment. All single clouds have been subsampled at 5mm uniform point-to-point space and merged, deleting duplicate points. The final point model is composed of 597 619 636 points and has been exported in two different versions in LAS format: i) X, Y, Z, R, G, B values for the point cloud with real colours coming from scanner cameras and ii) X, Y, Z, R, G, B values for the point cloud with intensity scalar field transformed into its colour components.
The two different LAS files of 14.4 Gb are imported inside the Holoverse Presenter environment and this process required 50 minutes. Inside the software, it is possible to set up the position of the cloud model in relation to the Hologram table digital reproduction, simulating the projected location/orientation that the cloud will have after display. Once the user is satisfied with the disposition of the 3D content, it is possible to convert the LAS model imported to the Euclideon UDS proprietary file format. Export time reached 5 minutes. It is fundamental to note that software export capabilities are directly correlated to the quantity of RAM installed on the local machine. For our test case, the maximum file dimension that could be imported from a LAS file to Holoverse present is 14.4 Gb; after this limit, the import process fails to return a software error. The display of the Santa Maria delle Grazie complex proved the Hologram Table can manage and render a huge dataset with great details without problems (Figura 6).  However, for the holographic presentation (as well as for most point cloud viewer and exploration systems), a high-quality visualization can be performed after cleaning the raw point model, deleting all bad points from each scan station. Indeed, distant points and low-quality measurements (e.g., taken with a high laser scanner incidence angle) will result in an irregular line of points, ruining the quality of the final point model (Figure 8).

The Ghesc village
One of the main and most successful visualizations that can be performed on the Hologram table is the display of environmental data. The experiment has been conducted selecting a UAV dense cloud of the village of Ghesc, composed exactly of 17 865 774 points. The LAS file size is 0,442 GB. It can be exported directly from the photogrammetric software (Agisoft Metashape used for the elaboration). However, the model was edited in the CloudCompare to clean bad points and verify the LAS attributes (X, Y, Z, R, G, B values). The import time of the point model in the Holoverse Present software required 30 seconds and 10 more seconds to convert to the proprietary UDS file format (Figure 9). The holographic device permits different point cloud render modes. Very interesting and efficient for this case-study results the DEM (Digital Elevation Model) visualization. This type of representation makes the quota and heights of buildings and ruins easily distinguishable and legible (Figure 10-11).

The 727 Cuneiform tablet
The clay tablet number 727 provided the best example to test the ability of the Hologram Table not only to display architectural or environmental data but also small objects very rich in details. For this purpose, the mesh model resulted from the abovedescribed scanning process has been used. The digital clone of the cuneiform tablets (STL file format and 0,123 Gb file size) with great details and high accuracy has been imported inside Holoverse Presenter software (20 sec of importing time).
Since it is a mesh model, the software does not require to convert the model into a UDS format, which is specific for point cloud objects. However, its import inside Holoverse Present is helpful to prepare the model view and, above all, to scale the reconstruction referred to the device physical size. Figure 12 shows the tablet hologram on the table flat screen at a graphic magnification scale (the physical object measures 3x5x7cm approximate). The holographic and enlarged visualization of these small objects with a surface very rich in detail proved useful for the study of the different facets that results from the cuneiform engravings. Some render mode allows the light direction simulation to reveal details that would be hidden from view in normal lighting condition. Usually, the same process is performed on the real object. For a correct identification and deciphering of the signs, it is necessary to change the light source; therefore, the tablet is manually rotated in different directions and at different angles. The aim is to identify the most effective lighting to bring out the signs imprinted in the clay (Surdi, 2017). The same manual inspection can be simulated with the holographic device that allows the change of the light direction using the wand controller. The changing of light direction projects different shadows on the mesh surface, showing new details ( Figure 13).
Unfortunately, the parametric model that governs lights projections does not work with point cloud that results uniformly illuminated. Figure 12. 727 cuneiform tablet holographic visualization.

CONCLUSIONS
This first experiment to visualize a big point cloud model with a holographic table outline great potentiality. Navigating the hologram is impressive fluid even when zooming to view a greater detail level, despite the high number of points. The power and performance of the 3D rendering engine of the point cloud are high and impressive. Some holographic representation and rendering, such as the DEM for the Ghesc village and variable light direction for the tablet, resulted in interesting applications.
To conclude two aspects can be highlighted. The first is that cleaning the terrestrial laser scanner point cloud significantly increases the visualisation quality. The second is that for huge models, a waving of the hologram can be observed. However, more complex, and time-consuming is the standard pipeline from raw terrestrial laser scan data to the global point cloud. Despite the significant reduction of on-site survey times and in-office processing times guaranteed by the VIS (Visual Inertial System) technology of the RTC 360 and the cloud-tocloud registration mode, this first step is more time-consuming. In fact, the management, editing and export of a significant number of scan files with high resolution takes a long time. On the other hand, the survey times are proportionally more significant for the image-based techniques and triangulationbased scanners. Still, the workflow that leads to the hologram is more streamlined. A future aim is to investigate tools and possible applications of holographic content into the Unity environment, which could offer more interaction with the hologram model. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China