AUTOMATIC RECOGNITION OF A PIPING SYSTEM FROM LARGE-SCALE TERRESTRIAL LASER SCAN DATA

Recently, changes in plant equipment have been becoming more frequent because of the short lifetime of the products, and constructing 3D shape models of existing plants (as-built models) from large-scale laser scanned data is expected to make their rebuilding processes more efficient. However, the laser scanned data of the existing plant has massive points, captures tangled objects and includes a large amount of noises, so that the manual reconstruction of a 3D model is very time-consuming and costs a lot. Piping systems especially, account for the greatest proportion of plant equipment. Therefore, the purpose of this research was to propose an algorithm which can automatically recognize a piping system from terrestrial laser scan data of the plant equipment. The straight portion of pipes, connecting parts and connection relationship of the piping system can be recognized in this algorithm. Eigenvalue analysis of the point clouds and of the normal vectors allows for the recognition. Using only point clouds, the recognition algorithm can be applied to registered point clouds and can be performed in a fully automatic way. The preliminary results of the recognition for large-scale scanned data from an oil rig plant have shown the effectiveness of the algorithm.


INTRODUCTION
Recently, in chemical, material and food plants, because of the short life cycle of the products in the market, changes in the plants' equipment have been becoming more frequent.However, the former changes are not necessarily recorded in the plant drawings in many cases in Japan.For this reason, unintended collisions between the existing equipment and the designed ones often occur in the construction stage, and this causes delays of the work and additional costs.On the other hand, with improvement of terrestrial laser scanner performance, massive point clouds of real objects have been acquired very easily and quickly.Also, there is a strong possibility that 3D models of production facilities and plant equipment could be reconstructed from these point clouds.
Once the models are built, the unintended collisions between the existing equipment and the designed ones can be efficiently checked and avoided before the construction stage.Therefore, 3D modelling of existing plants from scanned point clouds is considered to be an imperative process in the recent plant rebuilding process.However, the laser scanned data of the existing plant has a huge number of points, captures very tangled sets of objects and includes a large amount of noises.Therefore, recognizing each individual object from these tangled, enormous and noisy point clouds and building 3D models of them becomes nearly impossible or very time consuming when doing it in a manual way.Thus, automating recognition and construction of the 3D models from the point clouds needs to be strongly promoted in plant engineering.Chemical, material, and food plants consist of many types of objects.Among them, the piping system especially accounts for the greatest proportion.As shown in Fig. 1 and 2, a piping system consists of various piping elements and connection relationships; straight pipes, connecting parts, such as Tjunctions and elbows, indicators, valves, etc.Also, the connection relationship defines the logical connectivity between these piping elements.Several research studies have been proposed for algorithms of recognizing a piping system from laser scanned point clouds.However, these algorithms could not recognize pipes in a fully automatic way, or could not be applied to a registered point cloud, or could not recognize pipe parameters such as radii or length.Therefore, the purpose of our research was to propose a new algorithm that can automatically recognize piping elements and their connection relationships from a registered laser scanned point of a plant.In addition, the algorithm can recognize pipe The algorithm was tested for a large-scale laser scanned point cloud of a real plant, and the accuracy of the radii and their connection relationships were verified.

RELATED WORKS
So far, several research studies have been done for developing algorithms to recognize objects from terrestrial laser scanned point clouds.Binert proposed an algorithm which can recognize tree trunks from a point cloud measured from forests (Binert et.al 2006).Luo proposed one which recognizes pillars from a point cloud of building indoor environments (Luo et.al 2008).In both of these, objects are recognized by fitting circles to the points projected onto a horizontal plane.However their algorithms can only recognize column-like objects whose inclination was limited to nearly only the vertical one.Several research studies have also been done to recognize objects from laser scanned data of plant equipment.Masuda proposed an algorithm which can recognize planes and cylinders from the data of plants (Masuda et.al 2009).However, a region to be recognized has to be selected manually in advance.Matsunuma also proposed an algorithm similar to Masuda's (Matsunuma et.al 2010), but it requires the combination of measured point clouds, range imageries and brightness imageries.Rabbani also proposed an algorithm which reconstructs a 3D plant model from the combination of point cloud data and a photographic image which are taken from a single measuring location (Rabbani et.al 2004).However, the recognition algorithms can only be applied to the combination of the point cloud and the image generated by a single scan.Piping systems usually occupy a large-space in factories and multiple scans must be taken and each be registered to collect the point cloud which can cover the existing space of all the pipes.Unfortunately, it is hard for the above algorithms to be applied to a huge point cloud which is created as a result of the registration of the points generated by multiple scans.Andrew proposed an algorithm which can semi-automatically build a 3D model by matching a point cloud to a CAD model using spin images (Andrew et.al 1997).
However, the matching is inefficient because it uses an exhaustive search, and an experimental verification was not done.Bucksch proposed a skeletonization algorithm which classifies the scanned point cloud into several groups, each of which corresponds to a single skeleton.It can be used to extract the feature structure and to recognize the connection relationship in a piping system (Bucksch et.al 2010).However, it is hard for this algorithm to recognize straight portions of the pipe from the point cloud.
Recognizing points on pipes and model fitting to the pipes A1 •Points of piping system Recognizing connection relationship A2 •"pipe points" •radii of pipes •positions of pipes •"non-pipe points" •Connection relationship of piping system •Axis segments of pipes

ALGORITHM OVERVIEW
As shown in Fig. 3, our algorithm consists of two steps.
In the first step (A1), points on straight pipes, here after referred to "pipe points," are extracted, and their radii and the positions of the pipes are recognized as pipe parameters from a point cloud.Also, axis segments of the pipes are calculated from the extracted points and from the parameters.Then, in the second step (A2), the connection relationships among the extracted straight pipes are recognized using the positions of the axis segments.
The point clouds from the laser scan in a plant generally include, not only the points on the piping system, but also those on many other classes of objects, such as building columns, supporting brackets, containers, etc.However, in this research, in order to simplify the recognition, it is assumed that the points measured from objects other than the piping system are roughly removed by manual pre-processing and that the input point cloud to the recognition algorithm only includes the points of the piping system.The detailed processes of steps (A1) and (A2) are shown in sections 4 and 5.

RECOGNIZING POINTS ON PIPES AND MODEL FITTING TO PIPES
In this section, the detailed process of the recognizing points on pipes and fitting models to pipes are described.The flow of the process is shown in Fig. 4.

Estimating normal vectors (A11)
A scanned point cloud consists of a set of vertices.In order to estimate the normal vector at a vertex i , first, the covariance matrix i S is calculated by equation (1) (Qian et.al 2008), (Leo  et.al 2007) .
a set of neighbouring vertices contained in the sphere of radius 1 r centred at i . i v a position vector of i .
Then, the eigenvalues ) 0 ( , , are obtained by eigenvalue analysis of i S .If we set 1 r to smaller than the radius of pipe R , the vector 3 e will approximate the normal at i .Thus, 3 e is determined to be the initial normal vector at vertex i in the original coordinate frame z y x   (Mizoguchi et.al 2007).
a local coordinate frame whose origin is at i , and w axis parallel to i n .

Extracting points on pipes (A12)
First, a normal tensor i T at vertex i is calculated by equation (3) from a set of points ) , ( 3 r i N (Qian et.al 2008), (Leo et.al 2007).
Then, the eigenvalues ) 0 ( , , ˆ3 2  becomes much smaller than 1  and 2  .While at a vertex on an elbow or a T,Y junction, the normal vectors are distributed three-dimensionally.Then, 3  becomes larger than that of the former case.Thus, limiting 3  enables whole scanned points to be classified into "pipe points" and "non-pipe points".The threshold value is determined by the discriminant analysis method [William et.al 1980].

Grouping "pipe points" (A13)
After finding the pipe points, the points on one straight portion of a pipe are integrated into a single region by applying the region growing method.First, a seed point s of a region is chosen from "pipe points" at random.Then, other pipe points contained in a set of neighbouring points ) , ( 4 r s N centred at s are added to the region.Each of these added points are then chosen as a new seed point, and other "pipe points" adjacent to each seed point are added to the region.The above steps are repeated until "pipe points" exists in neighbourhoods of the seed points.

Extracting initial pipe parameters (A14)
After grouping the "pipe points," the axes of the pipes and their radii are estimated.First, a normal gauss image is created from all "pipe points" in a region.Then, as shown Fig. 5 (a), a plane is fitted to the gauss image using RANSAC, and an initial axis vector init p is estimated as the normal vector of the plane.In the RANSAC, the number of sampling is 500 and the outlier threshold is 0.01 in this paper.Then, all "pipe points" in the region are projected to the plane, and a circle is fitted to the projected points by the least-square method.As a result, the centre point and the radius of the fitted circle respectively become an initial point on the axis init v of the pipe and an initial radius init r of the pipe.Then, a cylinder is precisely fitted to the points of the region using the non-linear least square method (Shakarji et.al 1998) is components of a position vector p of a point on the axis, and r is the radius of the cylinder, the objective function of the fitting is shown in expressions (4), ( 5) and ( 6).
where  ) , , ( a position vector of the "pipe points". N number of the "pipe points".
Then points in the region are projected to the cylinder axis, and two end points of the projected points are determined as edge points e e  , , as shown in Fig. 7.The segment connecting e with e is used for recognizing pipe connecting relationships, as described in Section 5.

Correcting the pipe parameters (A15)
In the output of A12, "non-pipe points" still includes a few numbers of points on the straight portion of pipes due to the limitation of the discriminant analysis.For the improvement of fitting and recognition accuracy of pipes, the miss-classified points included in the "non-pipe points" are extracted from the set and are added to the "pipe point", and cylinders are re-fit to the "pipe points".In this step, as shown in Fig. 6, a seed point s which exists nearest to a cylinder originally fit to a pipe is chosen from "non-pipe points".Then the "non-pipe points" within the distance 1 Th from the cylinder are only selected from the points ) , ( 4 r s N . The selected points are then added to the pipe region and a cylinder is fitted once again.Then these selected points become new seed points.The above steps are repeated until a "non-pipe" point which lies within the distance 1 Th from the fitted cylinder exists in the neighbourhood of the new seed points.

RECOGNIZING CONNECTION RELATIONSHIP
In order to recognize the straight portions of a pipe occluded by other pipes and to identify junctions among the pipes, the connection relationship between pipe regions is recognized by checking the relative positions and orientations of the axis segments created in the Step A14.As shown in Fig. 7, an axis segment i S of a pipe has a centre point i p , two edge points i i e e  , , outward unit axis vectors i i u u  , and a radius of the fitted cylinder i R .Based on the segments, occluded portions of pipes are interpolated, and T and Y junctions and elbows are identified progressively.

Recognition of occluded pipe portions
In order to correctly interpolate an occluded pipe portion between two pipe segments, the following checking process is done.First, pairs of nearly collinear axis segments i S and j S which satisfy equations ( 7) and ( 8) are selected.
where  ij p a unit direction vector of a line connecting two segment centre points

Recognition of T,Y junction parts
After the interpolation of occluded straight pipe portions, T and Y junctions which connect multiple pipes are identified.As shown in Fig. 10, first, a pair of segments which satisfy the following equations ( 9) and (10) are selected.

Recognition of elbow parts
After the interpolation of T and Y junctions, elbows which connect two pipes are identified.As shown in Fig. 11, first, a pair of segments i S and j S which satisfy equation (10) are selected.Moreover, if the line segments i i e q and i j e q do not interfere with the other pipe and give the minimum distance

Classification of connecting elements
After inserting the new segments between the existing pipe segments according to 5.1-5.3, each edge point i e is finally classified into one of the five types using the following criteria:

1) Elbow
At the point, two segments connect, and the axis direction vectors .

2) Reducer
At point i e , two segments connect, i v and j v satisfy the condition , and the difference in radii , where r  is set to 0.5 times of the greater radius of i R or j R .

3) T-junction
At point i e , three segments connect, and the inner products among satisfy equation ( 11), where 1 .0

4) Y-junction
At point i e , three segments connect, and the inner products among do not satisfy equation (11).

5) Occluded portion
At point i e , two segments connect, the i v and j v satisfy the condition , and the difference in radii , where ) , max( 5 .0

RESULTS
As a preliminary experiment, point clouds were scanned from a Y junction (#points: 40,634) and an elbow (#points: 27,668) of PVC pipe using a 3D non-contact digitizer (KONICA MINOLTA VIVID 910).The thresholds were   .1.The maximum error was less than 0.60mm.Given the radius interval of the PVC pipe product series, recognition accuracy of the pipe parameters in our proposed algorithm is appropriate.
As an experiment for more realistic recognition, a set of complex point clouds were scanned from a real oil rig by a laser scanner (Cyra Technologies CYRAX2500), and they were registered to be one huge point cloud, as shown in Fig. 14.It had about 1,720,000 points.The total recognition process took 6311.8sec in PC (Core-i7) which includes normal vector calculation 2749.8sec and that of the normal tensor 2295.8sec.The thresholds were   .2.In the result of junction elements recognition, some elbows were mis-recognized as Y-junctions.

Table.2 Results of connecting parts recognition
The reason for this mis-classification was that the length of the recognized cylinder was over estimated and they were misidentified as Y junctions.Moreover, in the results, some elbows and T junctions were recognized as not being connection parts.The reason was that the occluded pipe portions were too long to be interpolated, and they were recognized as being separate.

CONCLUSIONS
A new algorithm was proposed that can automatically recognize a piping system from scanned point clouds.The points on pipes could be recognized by the normal tensor and the discriminant analysis method, and the radii and positions of pipe segments could be recognized by cylinder fitting using the non-linear least square method and region-growing.The connecting relationship between pipes could be classified based on the relative positions and orientations of segment axes of the fitted cylinders.The recognition results and the dimensional accuracy of the recognition were verified by smallscale and large-scale examples, and the results show the potential effectiveness of the proposed algorithm.
Of course, the algorithm still lacks a function which can discriminate point clouds included in the piping system from the other points in scanned data.The pillar recognition algorithm proposed by Luo might be of some help to the discrimination function (Luo et.al 2008).This remains to be part of our future work.

Fig. 2
Fig.1 Elements of a piping system

FigFig. 4
Fig.3 Recognition Algorithm Overview eigenvalues and eigenvectors show spatial distributions of the normal gauss image as shown in Fig 5.For example, at a vertex i on a straight portion of pipe, the normal vectors in ) , ( 3 r i N exist almost on one plane.Then 3

Fig. 6
Fig.5 Normal gauss image and eigenvectors of normal tensor be 0.98 in our experiments.Next, the possibility of the collision of a line segment j i e e with another pipe region is checked.As shown in Fig.9-(a), if the other pipe interferes with segment should not be interpolated as one pipe.On the contrary, as shown in Fig.9-(b), ifl is greater than the threshold l  , the other pipe does not interfere with distance is finally selected as an appropriate one to be interpolated, and a new pipe segment our experiment, which gave good recognition results.Moreover, if the line segment j i e q does not interfere the other pipe and gives the minimum distance process took 124.2sec for the Y-junction and 25.5 for the elbow.The recognition results of pipe point regions are shown in Fig.12, and those of the cylinder fitting and the connection relationship in Fig.13.The recognitions and the classifications were successfully done.The dimensional errors in outside radii between the recognized parameters and the real ones are shown in Table . The final recognition results of cylinder fitting to the pipe portions and of the connection relationships are shown in Fig.15and 16.The recognition accuracy of the connecting pars is also shown in Table Fig.7 An axis segment of pipe

Fig. 14 Fig
Fig.14 Scanned data of an oil rig