POINTO-A LOW COST SOLUTION TO POINT CLOUD PROCESSING

With advance in technology access to data especially 3D point cloud data becomes more and more an everyday task. 3D point clouds are usually captured with very expensive tools such as 3D laser scanners or very time consuming methods such as photogrammetry. Most of the available softwares for 3D point cloud processing are designed for experts and specialists in this field and are usually very large software packages containing variety of methods and tools. This results in softwares that are usually very expensive to acquire and also very difficult to use. Difficulty of use is caused by complicated user interfaces that is required to accommodate a large list of features. The aim of these complex softwares is to provide a powerful tool for a specific group of specialist. However they are not necessary required by the majority of the up coming average users of point clouds. In addition to complexity and high costs of these softwares they generally rely on expensive and modern hardware and only compatible with one specific operating system. Many point cloud customers are not point cloud processing experts or willing to spend the high acquisition costs of these expensive softwares and hardwares. In this paper we introduce a solution for low cost point cloud processing. Our approach is designed to accommodate the needs of the average point cloud user. To reduce the cost and complexity of software our approach focuses on one functionality at a time in contrast with most available softwares and tools that aim to solve as many problems as possible at the same time. Our simple and user oriented design improve the user experience and empower us to optimize our methods for creation of an efficient software. In this paper we introduce Pointo family as a series of connected softwares to provide easy to use tools with simple design for different point cloud processing requirements. PointoVIEWER and PointoCAD are introduced as the first components of the Pointo family to provide a fast and efficient visualization with the ability to add annotation and documentation to the point clouds.


INTRODUCTION
3D data especially 3D point clouds and 3D models are used everyday to represent environments and objects.They are used in a variety of fields such as: industrial design, heritage documentation, city planning, entertainment industry, medical and many more.Modern computer platforms are capable of visualizing very large 3D data sets that provide a more realistic representation of items in comparison with traditional 2D methods such as drawings, images and sketches.The generation and use of 3D data is categorized into two categories.First category is the 3D data that is generated in order to manufacture or build a physical item based on that 3D model, this category is very common with architects and industrial designers.Second category is the 3D data that is generated from the existing physical items, capturing the 3D information of an old building for preservation and reconstruction of the environment or generation of 3D data based on an existing manufactured item for further study and perhaps reproduction of a similar item.The main difference between these categories is the need for data acquisition in the second category while in the first category the data is generated with the help of softwares and experts.The high costs and limitations of data acquisition step was the main reason that the photo realistic and detailed 3D model of existing environment were not very common.However, with the advance in technology and the rise of new hardwares and methods especially the low cost 3D point cloud processing methods the access to 3D point clouds and 3D models of the objects and environments has become easy and a common task in many fields particularly in archeology and heritage documentation.
Modern terrestrial laser scanners are capable of capturing measurements from an environment with an impressive rate, an aver-age of 35 to 40 million points are captured in 5 minutes.These scanners are divided into two categories based on the technologies used for emitting the laser beam.First category is the pulsedbased laser scanners.These scanners emit a short pulse of light and directly measure the amount of time it takes for the pulse to travel from the scanner to the object and back to the scanner.These scanners have lower acquisition rate compare to the second category, but they are capable of measurements of extremely high ranges up to 6 km.The second group of laser scanners emits modulated laser light at a known frequency.The change of the phase between the emitted light and the received signal determines the light travel time.Due to the limitation of phase shifts the maximum range of these scanners is inherently limited.Current phase-based laser scanners achieve a range of about 80-120m.However, since phase-based laser scanners emit the light continuously they can operate at higher measurement speed than the pulsed-based systems.Other than terrestrial laser scanners the hand held and close range scanners are the other group of 3D measurement devices.These systems are usually a triangulationbased active ranging systems.They use geometric properties of structured light to calculate the distance to objects from the scanner.Structure light is projected onto the environment and the scene is captured by an imaging device.The pattern is extracted from the image and based on the deformation and geometric values the triangulation system calculates the 3D coordinates.
Besides the distance to objects, laser scanners determine the intensity of the reflected light.This yields information about the reflectivity of the scanned surface.Further combination of Light Detection And Ranging (LiDAR) systems with color cameras incorporate the color information of the measured surface into the point cloud data.Modern laser scanners commonly incorporate the color camera into their system.High Dynamic Range (HDR) The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W8, 2017 5th International Workshop LowCost 3D -Sensors, Algorithms, Applications, 28-29 November 2017, Hamburg, Germany cameras are used to produce images with a greater dynamic range of luminosity than with the standard digital camera.This method is used to generate point clouds with a better and more realistic color information of the environment containing both very bright light and shadows.In addition to color cameras, thermal cameras can be mounted on the scanning unites.After proper calibration between the cameras and the laser scanner the thermal images can be used for generating the 3D thermal map of the environments and objects.
Similar to laser scanning methods cameras are used directly for generating point clouds.Photogrammetric methods such as Structure From Motion (SFM) are used to calculate the depth from from 2D images.This technique estimates the 3D structure of objects and environment from a sequence of 2D images.Similar to the problem of solving the estimation of 3D structure from a stereo vision the SFM method utilizes the feature detection methods such as Scale Invariant Feature Transform (SIFT) (Lowe, 1999;Lowe, 2004) and Speeded Up Robust Features (SURF) (Bay et al., 2006;Bay et al., 2008) for the detection of correspondences between 2D images and the 3D reconstruction of the objects.This method is capable of generating 3D point clouds of environment from a series of images that can be captured with a digital camera, therefore, after the training and acquiring the skills the 3D point clouds can be generated with a relatively low priced hardware.
With all these acquisition methods and the continues reduction of the cost of point cloud generation due to the advances in technology and the accessibility to more hardware and software systems, point clouds have become the standard method of representation of existing models and environments in 3D for many fields especially archeology and cultural heritage documentation.
They have replaced traditional methods of documentation such as 2D images, hand drawings, sketches and models of archeological excavation and heritage sites.Due to the fact that point clouds contain a very dense representation of the environment they are a very detailed and a more realistic representation of the acquisition environment in comparison with traditional hand drawings and 2D images.Point clouds provide this photorealistic information with a short acquisition time.However, software packages are required to provide tools for visualization, documentation, modification and interaction with 3D point clouds.In the following sections an overview of point cloud processing methods is presented and limitations of point clouds and modern software systems are stated.Afterwards, our solution to overcome these limitations is proposed and it proceeds to the introduction of our family of software packages coined as Pointo Family.Finally the first two parts of this family are introduced, namely PointoVIEWER and PointoCAD.

STATE OF THE ART
Availability of point clouds of objects and environments increases daily due to the reduction in the price of point cloud acquisition and the help of increasing number of existing methods and hardwares in the market.Additionally, a great amount of research is conducted on point cloud processing methods by experts in many fields owing to the diverse use case of point clouds, such as point cloud processing in the fields of Robotics, Computer Vision, Surveying, Automotive Industry, Entertainment Industry, Archeology and Cultural Heritage and many more.In order to provide the tools and necessary functionalities for all the aforementioned fields experts have developed variety of methods.
Point cloud representation is at the core of any point cloud processing method.Modern 3D measurement systems simplify the collection of large 3D point clouds.These systems are used to generate accurate and high resolution point clouds.Regardless of the acquisition method point clouds contain a very large number of points and therefore they require an efficient method of visualization.Captured scans from a 3D laser scanning unite contain range information from measured surfaces in the environment.
In addition to range information modern scanners capture the reflectance and color values of the measured surface.The captured environment can be imagined as a sphere with the measurement system positioned at the center of the sphere.The raw data from a 3D laser measurement system is commonly stored as x, y, z in the cartesian coordinate system.An alternative method is to represent the same information in the spherical coordinate system.The ✓, ', r represent the horizontal and vertical angles of measurement with the measured distance of r.This information is used for generation of panorama images from point clouds.Several methods of panorama generation has been studied in (Houshiar et al., 2015b).This introduces the 2D image processing methods in the field of 3D point clouds.The large point clouds acquired by modern terrestrial 3D laser scanners demand large computer memory.To process them in the main memory of a standard system many methods have been proposed in the literature.
Point clouds contain only the surface information of the objects in the environment and are not really a volumetric data.Therefore, methods that rely on spacial data structures are often used to represent the point clouds, such as k-d tree and octree.(Bentley, 1975, Meagher, 1982) The basis of many point cloud processing methods is the ability to search and select points in a large point cloud in a short time.These methods are essential for the fast visualization of 3D point clouds.
Point cloud registration is another method that rely heavily on the ability of efficient search of points in point clouds.These methods focus on the process of merging point clouds of an environment that were captured from different viewing angles.This is a common method to combine many scans captured with laser scanning unites.Multiple scans are required in order to capture a complete environment without any occlusion.Point cloud registration methods are divided into feature-based and featureless methods.The latter relies directly on the points in the point cloud.Iterative Closest Point (ICP) (Besl and McKay, 1992) is a renowned method for feature-less registration of point clouds.
The main drawback of ICP method is the requirement of a good initial estimate of the transformation between point clouds in order to converge into a correct result.Several approaches can be used to provide this initial estimate.Global Navigation Satellite System (GNSS), local tracking systems, odometry information of mobile robotic platform and many more approaches are available.
The former registration group relies on the detection of features and markers in the point clouds.These features are either manually positioned artificial markers in the environment or natural features of the environment.Several feature detection methods have been studied in the literature for detecting natural features directly from 3D point clouds (Barnea and Filin, 2008, Brenner et al., 2008, Magnusson et al., 2009, Pathak et al., 2010) In addition to 3D feature detection with the help of panorama image generation, all 2D image feature detection methods can be used for the detection of features in point clouds, such as SIFT and SURF feature detection methods.(Houshiar et al., 2015b) presents a detailed study of the use of 2D feature detection on panorama images generated form point clouds for the purpose of feature-based point cloud registration.In case of lack of positioning information during the data acquisition feature-based registration is the only method that can provide the required initial estimate for the ICP algorithm.
Another aspect of the point clouds that is critical to efficient point cloud processing is the large size of the data.Fast data acquisition The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W8, 2017 5th International Workshop LowCost 3D -Sensors, Algorithms, Applications, 28-29 November 2017, Hamburg, Germany rates generates many point clouds with large number of points that is even difficult for modern computer systems with powerful processing units and large memory to process.Therefore, methods of reduction and compression of point clouds have gained the attention of researchers.As mentioned earlier registration process is an important step but time consuming part of point cloud processing.Data reduction becomes necessary to efficiently perform this operation.The reduced point cloud will notably increase the speed of registration.There are approaches to reduce the complexity of scan registration by extracting planes from the data (Brenner, 2000;Pathak et al., 2010).These methods by using geometrical features neglect the precise point measurements of point clouds.Other methods of point cloud reduction separate the space into voxels and sample the points from each voxel.Efficient methods with uniform voxels use octrees (Meagher, 1982).Some methods rely on range images or panorama images generated from point clouds for point reduction (Mandow et al., 2010;Houshiar et al., 2013a).These reduction methods are necessary to increase the registration efficiency.However, the original and complete point clouds are usually preferred for visualization and other processes.Large point clouds and 3D data are archived and transmitted across networks for data sharing and collaboration between users.Data captured with LiDAR systems are usually stored in the vendor specific binary files.3D data captured with other methods and especially for exchange between users the simple ASCII representation is used.ASCII formats represents the data in lines containing the attributes of each measurement.This is flexible and very easy to understand, especially for humans.However, storing millions of 3D points in a textual format generates very large files for which only insufficient parsing capabilities exist.The American Society of Photogrammetry and Remote Sensing (ASPRS) created a simple binary exchange format called the LAS format (American Society of Photogrammetry and Remote Sensing, 2017).Furthermore, they introduced the LAZ as a lossless compressor for LiDAR data stored in the LAS format (Isenburg, 2012).Several other compression methods have been proposed in the literature based on tree structures, such as k-d tree and octree (Devillers and Gandoin, 2000;Huang et al., 2006;Elseberg et al., 2013;Hornung et al., 2013).With the use of panorama generation methods based on LiDAR data, conventional image compression methods are applied to compress 3D point clouds (Houshiar and Nüchter, 2015).
To solve the few aforementioned point cloud processing tasks and many other tasks, several point cloud processing libraries and softwares has been developed.Point cloud processing libraries, such as the 3D Toolkit (3DTK) (N¨uchter and et al, 2017) and Point Cloud Library (PCL) (Rusu and Cousins, 2011; Rusu and et al, 2017) provide many methods and algorithms for efficient point cloud processing.However, the focus of these systems is on software developers and scientific researchers.Due to their limited and complicated user interface and providing the majority of their functionality only through the Application Programming Interface (API), these systems are not intended for average user.Other options for the user to access several methods for their point cloud processing needs are open-source softwares or expensive proprietary software packages.

HIGH COST OF MODERN SYSTEMS
With advance in technology access to data especially 3D point cloud data becomes more and more an everyday task.3D point clouds are usually captured with very expensive tools such as 3D laser scanners or very time consuming methods such as photogrammetry.Generally data acquisition is not the final goal and user is required to utilize several point cloud processing methods to achieve their final goal, such as registration of multiple point clouds to represent an entire environment, categorization and classification of point cloud to add semantic to the point cloud, add annotation and drawings directly on point clouds to create a documentation for the data.To achieve these experts utilize available software packages or develop their own software tools based on libraries and toolkits available for software developers.On the other hand the average user can only use the opensource or commercial software packages.Most of the available softwares for 3D point cloud processing are designed for specialists in this field and are usually very large software packages containing variety of methods and tools.This results in softwares that are very expensive to acquire and also very difficult to use.Difficulty of use is caused by complicated user interfaces that is required to accommodate a large list of features.After the acquisition of the expensive software users require to complete a training period to understand the functionalities of the software and learn and create a workflow that helps them to achieve their goals.The aim of these complex softwares is to provide a powerful tool for the specific group of specialist that need multiple functionalities and also are working mainly with point cloud data to provide services for others.These softwares however, are not necessarily required by the majority of the up coming average users of point clouds.The average users utilize point clouds as a supplement to their work, such as architects and archeologies, and not directly producing point clouds and working with them.Therefore, the high costs of purchase and training associated with complex software packages are not desirable for them.
In addition to complexity and high costs of these softwares they generally rely on expensive and modern hardware.They are designed to work on hardware with a very large internal memory and a very powerful graphic processing units.In the best case scenario these softwares are capable of using all the power of modern systems and are not only dependent on powerful hardware due to their lack of efficiency in implementation and use of inefficient algorithms.In addition to hardware limitation, these softwares are only compatible with one specific operating system which imposes more limitation on the user.Many point cloud customers are not point cloud processing experts or willing to spend the high acquisition costs of these expensive softwares and the costs associated with specialized hardware for their point cloud processing needs.One solution for this group is to utilize free and open-source softwares.Despite the fact that opensource software eliminates the cost of software acquisition, unfortunately these software packages usually contain many bugs and are not optimized for best user experience.Generally they all follow the same principle as commercial softwares and aim to solve all point cloud processing needs with one and only one software which directly influences the complexity of a software.Therefore, the necessity of training and the cost associated with it remains as a disadvantage for these softwares.Other issues with open-source softwares are often the lack of customer support and limited future development that makes these systems unreliable for longterm usage.Furthermore, they still rely on very powerful and specialized hardware and sometimes they are only available on one type of operating system which again further limits the user.

INTRODUCING POINTO FAMILY
The goal of our approach is to reduce the cost of point cloud processing with special designed softwares that have low level of complexity compare to other point cloud processing softwares and as a result have a lower required cost of training.To achieve the goal of our approach we focus on only one feature at a time to be able to reduce the complexity of the software.This permits a lower price of development which results in softwares with The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W8, 2017 5th International Workshop LowCost 3D -Sensors, Algorithms, Applications, 28-29 November 2017, Hamburg, Germany lower prices for customers.Focusing on one major functionality guarantees the simplicity of the software that in turn provides faster learning process.The simplicity is important both for developer and user.A well designed software that does not have to solve all the problems together is optimized for the one major task and provides an efficient solution to that problem.An efficient and simple software for user means no extra training time and means that the average user does not have to deal with hundreds of settings and parameters without fully understanding the purpose of them.We introduce point cloud processing to general/average user and aim to reduce user expenses by limiting the need for experts and the required time to master the software.General point cloud processing software packages in addition to have a very high cost tend to require a very expensive and powerful hardware.Our approach with focus on only one feature at a time can reduce the cost of the required hardware by optimizing the software to perform its one major task efficiently.We concentrate on efficient point cloud processing methods for each feature to reduce the hardware requirements.This eliminates the need for expensive hardware that is acquired only for the use of an even more expensive software package.Our approach works toward the use of available average office hardware even for the lower end hardware available in public administration offices.Our multi platform development strategy empowers the user to choose any major operating system available for desktop computers, namely Windows, Linux and MacOS.They can choose the operating system that they are more comfortable with rather than using an operating system that is a requirement of the software.
Most users only require some simple and specific functionality to perform on point clouds and can not afford the expensive requirements of current point cloud processing softwares.This implies that for every simple and small processing need, the user requires the help and expertise of another company, most probably the company that provides the user with the original 3D data.Every request or interaction on the point cloud will create additional cost for the average user that might have not been foreseen in the initial budget of the project.Our approach with the introduction of the Pointo software family fills the gap between 3D point cloud experts and average user and provides a cheaper solution for the user.The request for new functionality or software can be provided directly from the user and the modularity of our software family indicates that user has the option to choose only functionalities that they require.Based on this methodology we designed Pointo family as series of softwares that provides a low cost solution to point cloud processing.The development of all softwares in this family will follow a certain design guideline both to guarantee the lower cost of development and to provide a familiar interface for the user that utilizes multiple softwares of this family.In our approach user interaction and user interface of these softwares are considered parts of the more important aspects of the development.Our development method is very user oriented and we are of the opinion that user experience with a software system is as important as the functionality of the system.We achieve this with using a simple and user friendly design guideline.All softwares in this family follow the same design guideline and will provide only one major functionality.To establish a smooth workflow between different softwares, a shared project is used between softwares and the newly generated data from each software is added to the same project and if necessary available to other softwares.
The project was granted from 2012 to 2014 by the German Federal Ministry of Economics and Technology (BMWi) to solve the three main issues of archeological documentation process, namely the reduction of the required time for documentation, provide accessibility of 3D data directly on excavation site for the archeologist and to increase the quality of the documentation by working with point clouds and adding semantic information into them.In this paper we introduce PointoCAD as the first software of the Pointo software family.PointoCAD is developed to assist especially archaeologist with data generation based on 3D point clouds during the documentation process of an excavation.PointoCAD includes PointoVIEWER as an efficient point cloud viewer.PointoVIEWER will be included in all future Pointo softwares since the point cloud visualization is the main requirement of any point cloud processing software.

PointoVIEWER
Point cloud visualization is an essential part of any point cloud processing system.The ability to visualize the captured 3D points efficiently is the main focus of any point cloud viewer.Point clouds generally contain hundred millions of points.Fast processing of these large number of points and especially the visualization of these points require a data structure that provides efficient search and access to the points.In Pointo family we utilize the octree data structure.This is a data structure for indexing 3D data.The octree represents the data in a tree structure.The structure divides the 3D environment into rectangular cuboids.These volumetric cuboids are represented as octree nodes.The root node of the tree represent the whole data.Each node has up to eight children.child nodes correspond to octants of the cube.A node without a child implies that for the corresponding volume no further subdivision is necessary and the volume can be uniformly represented.Storing a point cloud in an octree has a stopping rule for the accepted volume.During the storing the point cloud the stopping criteria is defined as both maximal depth and minimal number of points per node.This criteria is applied to volumes without points, such that child nodes are created only for volumes that contain points.Point cloud points are stored in the leaf nodes of the octree.All nodes without points are considered as empty space.Since point clouds contain the surface information of the objects the 3D point clouds are not fully volumetric.The point cloud only represents the occupied space in the environment and most of the environment is free.This produces octree nodes with only few children.The octree data structure is therefore the ideal data structure for point cloud representation.
Figure 1 shows the octree data structure with a depth of three.
Point clouds in all Pointo softwares will be imported to an internal octree data structure.PointoVIEWER is a fast and easy to use visualization tool that is integrated into all Pointo family softwares to visualize point clouds in a matter of seconds (cf. Figure 2).This is achieved as the result of the octree data structure that allows for real time point density reduction.User can manually select the density of the visualized point cloud.We also introduce an automatic density reduction method to reach a minimum 30 frame per second visualization on any hardware.This improves the interaction of the user with the point cloud and is necessary for smooth navigation of the data.PointoVIEWER automatically calculates the required time for rendering each frame and according to this information reduces the number of points to a subset of point on the fly.Additionally it distinguishes between navigation and idle mode to provide a higher number of visualized points during the idle mode.Idle mode is defined as a state that there has been no navigation input from a user for certain period of time.Navigation mode is defined as a state that The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W8, 2017 5th International Workshop LowCost 3D -Sensors, Algorithms, Applications, 28-29 November 2017, Hamburg, Germany  and allow them to work with their data virtually on any available computer hardware.

PointoCAD
One essential point cloud processing task is adding documentation and semantic into the point clouds.The ability to add annotation and metadata to points is used in many fields that utilize point clouds on a daily basis.Archeological observations are commonly recorded on-site using conventional methods.The The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W8, 2017 5th International Workshop LowCost 3D -Sensors, Algorithms, Applications, 28-29 November 2017, Hamburg, Germany outline of findings are drawn on graph paper with detailed information with the help of an expert.Images, archeological notes and measurements are collected during the excavation that are of a great value in analysis of archeological findings in any project.However, the acquisition of these data with conventional methods is a very time consuming and demanding task.Due to lack of availability of the excavation after the data acquisition process any error or insufficiency in the captured data is not retrievable, hence the increase in the use of 3D data acquisition and computer aided systems.The captured 3D data is less prone to human error and is objective.Commonly the entire environment is captured during the acquisition process, therefore the original conditions of the excavation is preserved.The main feature of PointoCAD is to provide the possibility for the user especially the archeologist for labeling their findings on-site for further inspection.Through this process user classifies and marks the significant parts of the point cloud by annotating the features and findings of an excavation directly on the 3D point cloud.Additionally, the precise 3D measurements are provided by the extremely accurate point clouds that are generated by 3D laser scanners.In an collaborative environment an inspector can mark and annotate the areas that require further work.On conventional excavations archae-

CONCLUSION
Use of 3D data and point cloud processing needs are increasing rapidly.However, most available commercial solutions are only focusing on experts and are not design for average users.The available solutions are tend to be very complex and very expensive.They provide multiple features and functionalities which is the source of the increase in complexity of these softwares.High cost of acquisition and training for these softwares is only justified for experts.On the other hand the open-source community also provides solutions that eliminates the acquisition cost of the systems.However, these softwares follow the same principle of commercial solution and aim to solve multiple problems together.This puts open-source solutions in the same category of complexity as the commercial softwares, hence the cost and requirement of training is not eliminated.Other issues with open-source softwares is the lack of frequent update and bug fix and even sometimes the the lack of customer support and further development.
In this paper we presented our solution to overcome the high costs of current point cloud processing solutions by introducing Pointo family.The aim of our approach is to focus on one feature at a time to reduce the complexity of the software.This permits lower price for development which in turn reduces the acquisition cost for the average user.Our focus on one feature for each software in

Figure 1 .
Figure 1.Top: Spatial subdivisions of an octree up to level 3. Occupied leaf nodes are shaded as grey.Bottom: The corresponding tree structure of the sparse data structure.

Figure 4 .
Figure 4. Top: Representation of drawings on point clouds.Bottom: Representation of same drawings as an exported DXF file in a CAD system.
Pointo family allow us to increase the optimization and efficiency of the offered methods.The attention to design guidelines and elimination of complex interfaces strengthen the user experience and reduces or even eliminates the costs for training.Due to the modularity essence of a family of softwares instead of one large software, users are equipped with the option to choose and acquire only the functionalities that they require.With the introduction of Pointo family we also introduced the PointoVIEWER and PointoCAD as the first instalment of this software family.PointoVIEWER is an efficient and fast visualization tool that uses the octree data structure to achieve the smooth visualization of large point clouds on any hardware.PointoCAD is the first step into point cloud processing by providing the tool for adding documentation and annotation into point clouds.