3D SCANNING OF LIVE PIGS SYSTEM AND ITS APPLICATION IN BODY MEASUREMENTS

The shape of a live pig is an important indicator of its health and value, whether for breeding or for carcass quality. This paper implements a prototype system for live single pig body surface 3d scanning based on two consumer depth cameras, utilizing the 3d point clouds data. These cameras are calibrated in advance to have a common coordinate system. The live 3D point clouds stream of moving single pig is obtained by two Xtion Pro Live sensors from different viewpoints simultaneously. A novel detection method is proposed and applied to automatically detect the frames containing pigs with the correct posture from the point clouds stream, according to the geometric characteristics of pig’s shape. The proposed method is incorporated in a hybrid scheme, that serves as the preprocessing step in a body measurements framework for pigs. Experimental results show the portability of our scanning system and effectiveness of our detection method. Furthermore, an updated this point cloud preprocessing software for livestock body measurements can be downloaded freely from https://github.com/LiveStockShapeAnalysis to livestock industry, research community and can be used for monitoring livestock growth status.


INTRODUCTION
The shape, or conformation, of a live pig is an important indicator of its health and value, whether for breeding or for carcass quality.Currently most assessments of live animal conformation are carried out by eye or hand, and depend on the subjective opinion of the buyer or stockman.Shape plays a role in measuring the general health of the animal.In case of pigs, this is typically estimated from fatness, which is expressed as a condition score on a scale from 1 to 5 (Wu et al., 2004).The main method for determining the score is by touch, combined with a visual inspection of the shape as seen from the rear (Whittemore and Schofield, 2000).Weight can be estimated from shape measurements.Regular weighing is important because it allows the diet to be matched to the requirements of the pigs.The traditional method of weighing without scales is to measure the girth just behind the front legs with a weight tape, but researchers showed that it can also be estimated non-invasively from images or 3D point clouds (Li et al., 2014).
To measure body shape of pigs, we implement a portable 3D scanning system to acquire point clouds stream of live single pig body surface and propose one novel detection method to automatically detect the frames containing pigs with the correct posture from the point clouds stream, according to the geometric characteristics of pig's shape.Finally, the proposed method is integrated into a body measurements framework for livestock as the preprocessing step.
The overall structure of the paper is organized as follows.The next Sec.2gives a summary of the related work on livestock body shape measurements from point cloud data.In Sec.3 portable 3D * Corresponding author scanning system for live pig under farming circumstances is addressed.In order to measure body shape of pig, the novel detection method for recognition of frames with correct posture is presented in Sec.4.Sec.5 includes scenarios from agricultural applications and thus addresses the initially stated problem.Then we show results in Sec.6.Concluding remarks and prospective problems are discussed in the final section.

RELATED WORKS
The use of imaging or vision systems to predict or measure pigs body shape has been presented in several papers during the last 30 years (Wongsriworaphon et al., 2015, Kongsro, 2014).Vision systems based on visible light is often affected by variation in ambient lighting, and must be calibrated accordingly.Differences in animal color and subtraction of background is often a difficult task.Additionally, the lack of the third dimension in vision limits applications utilizing depth information.Photogrammetry stereo techniques have been introduced to measure farm animals in three dimensions(3-D) (Wu et al., 2004).However, these photogrammetric systems are difficult to implement under farming circumstances.Novel 3D scanning systems can solve the problems posed by conventional 2-D vision systems, including photogrammetry stereo techniques (Kongsro, 2014).As a result, there has been increasing demand for these techniques in livestock farming.Recently, consumer depth sensors based on a structured infrared-light(IR) system, such as the Microsoft Kinect (Weber et al., 2015)or the ASUS Xtion Pro, provide 3D data at low cost and have opened new possibilities for acquiring information of livestock conformation.Weber et al. have written the software for recording 3D images from TOF camera(SwissRanger SR4000, MesaImaging, Switzerland), taken several cuts along aligned pose of cow through a cow's surface in order to calculate traits that are meaningful to the surface's changes induced by varying body condition during lactation (Weber et al., 2014).But they only observed part body surface of livestock.A calibrated 3D reconstruction system for cattle using three Kinect sensors was introduced to 3D scan the whole body, the important traits for evaluating the shape and posture of the cow were estimated using the point cloud data aligned with tilt sensor (Kawasue et al., 2013).However, this whole livestock body surface 3D scanning system needs to be fixed in one room.Thus users have to pull the livestock into the right spot.However, this whole livestock body surface 3D scanning is time consuming and costly for farmers due to that user have to pull the livestock into the right spot in order to scan the whole body surface.On the other hand, high levels of stress can be induced by pulling the livestock into the right spot, especially in the case of younger livestock.
As increasing number of studies for shape measurement purposes in livestock farming, many software have been developed in order to acquire 3D surface data of livestock (Kawasue et al., 2013, Weber et al., 2014, Guo et al., 2013), to measure livestock conformation (Weber et al., 2014).However, many of these systems were designed only for scanning part of livestock body and state of art livestock whole body surface scanning systems were hard to deploy under livestock production circumstances.
In this work, our contribution is to implement one portable 3D scanning system for live pig under farming circumstances, propose one novel detection method to automatically detect the frames containing pigs with the correct posture from the point clouds stream, according to the geometric characteristics of pig's shape and apply it on the livestock body measuring pipeline successfully.
Figure 1.Capturing setup for scanning live pig

3D SCANNING SYSTEM OF LIVE PIG
A point cloud generated by an RGBD camera is a 2.5D description of the actual scene.Thus these kinds of data are unmeasurable due to occlusions from one view point.This motivated the usage of double RGBD cameras setup to achieve a full 3D description of a pig as shown in Fig. 1.This typical capturing setup would avoid interference issues (Macknojia et al., 2013) between two depth cameras due to the fact that two depth cameras face each other, but are occluded by the scene content.In the following subsections we describe how to scan the whole live pig with this setup.

Calibration tool for two RGBD camera(Xtion pro live)
In general, techniques for 3D reconstruction using RGBD images combine scans from a single moving camera.These techniques build a 3D model of the scene over time by aligning and concatenating successive scans (Zollhöfer et al., 2014, Hao andMayer, 2013).In contrast, our goal is to provide the reconstructed 3D non rigid object observed from two views simultaneously.To realize a system using multiple Kinects, it is one of fundamental and essential tasks to calibrate their cameras, that is, intrinsic parameters including their focal length and radial distortion and extrinsic ones consisting of their rotation and translation.Many automated calibration solutions were proposed (Staranowicz et al., 2014, Macknojia et al., 2013).However, those methods are heavily dependent on man made objects(checkerboard etc) which may be easily damaged under farming circumstances.More over, automated calibration solutions are not always be robust enough under different conditions.The work presented this section introduces a calibration tool we implemented for two Xtion pro live devices, which specifically addresses only external parameters that are the relative position and orientation between both depth cameras.Research shows that improvement of accuracy is limited by internal calibration of depth sensors.Therefore intrinsic parameters for the color and IR cameras and extrinsic parameters between them are obtained from SDK driver directly in our work.
The calibration procedure starts by acquiring two point clouds denoted by S = {si} and T = {ti}, from both depth cameras respectively.Then the relative position and orientation between both depth cameras can be estimated by aligning S and T .To do so, the calibration tool is designed to allow users choose multiple corresponding points from two point clouds which are then used to estimate the rigid transformation for alignment of two point clouds.The resulting set of correspondences is denoted by K = {(si, ti)} with si ∈ S and ti ∈ T .Based on the correspondences from the previous stage, the transformation supported by the correspondences is computed and it is refined using least squares fitting on all correspondences.
where Tst is the rigid transformation between the two point clouds and K is the set of all correspondences and (si, ti) ∈ K.
Note that in order to avoid switching between homogeneous and inhomogeneous coordinates when applying rigid transformations, we will treat Tst as a function that operates on points and applies a rotation R followed a translation c to them.The calibration tool further saves Tst as calibration file which will be used in the next stage.Note that calibration will be needed as long as the relative pose of both depth cameras is adjusted to fit the size of pig, and vice versa.Fig. 2 shows an example of calibration session by using our calibration tool.For quantitative results of calibration tool we refer the readers to our another published paper (Guo et al., 2014).

Software for streaming 3D point cloud realtime
To scan the live pigs, based on the calibration parameters from previous stage we have implemented a point clouds acquiring software framework for streaming 3D point cloud from both depth cameras simultaneously, which integrate the calibration procedure as one critical step.This software is a desktop application with graphical user interface(GUI).It is developed in C++.
The cross-platform application framework Qt is used for GUI related tasks.The Point Cloud Library (PCL) (Rusu and Cousins, 2011) is used for many of the supporting tasks, like point cloud saving, transforming, visualization and depth cameras operations.The block diagram illustrating the overall live pig 3D scanning process is shown in Fig. 3.The software starts by detecting available depth cameras.Then the coordinate system of first depth camera will be set as the global coordinate system.User could either load the calibration parameters estimated before or do the calibration by following the procedure in Sec.3.1 at one time.After that the point clouds acquired from second detph camera will be transformed into the global coordinate system.In order to scan the whole pig, the transformed point clouds will be merged with point clouds from first depth camera directly.Eventually, dynamic 3D point cloud sequences recorded from one moving pig will be saved.

CORRECT POSTURE DETECTION
3D scanning of live pigs is generally given in arbitrary posture in 3D-space.However, most of traits are measured only if that pig has the correct posture (Wongsriworaphon et al., 2015).Therefore, the input to this stage is dynamic 3D point cloud sequences from last section and the desired output is a set of frames with correct posture which will be used to take body measurements in Sec.5.We begin by downsampling the point cloud sequences using an octree with leaf size dr.At most one point per leaf is retained and all subsequent steps are applied on the donwsampled point clouds denoted by D = {Dt} where Dt = {p t i }.Where t is the frame index and i is the point index.Typically, 2r is used as the leaf size, where r is the resolution of the point cloud defined as the average distance between neighboring points.

Geometric definition of correct posture
When a pig is measured for body measurements, the animal needs to be standing with the correct posture; its dorsal line and cheek must make a straight line, and its four hooves must make a rectangle (Wongsriworaphon et al., 2015).To comply with these requirements, we make the following two assumptions about a pig with correct posture in geometrical sense.
1.When a pig stands naturally, its four hooves must make a rectangle.In other words, the line L3 formed by two front hooves of the pig is parallel to the line L1 formed by its two rear hooves, perpendicular to the lines L2,L4 formed by its two lateral hooves, as shown in Fig. 4. 2. A pig stands on the horizontal ground plane with its head forward.That is to say, the skeleton of pig top view is almost a straight line.
The first assumption is a requirement of the pig posture that make sure objective measurement in vertical direction.The second one is an assumption that ensure objective measurement in horizontal direction.Fig. 4 shows an example of qualified point cloud scanning in last section.

Segmentation of live pig automatically
Before we proceed to further processing, the segmentation of each frame Dt into pig and background is a necessary operation in the our workflow.To do so we firstly detect the ground plane based on RANSAC (Schnabel et al., 2007) as follows.The RANSAC paradigm extracts plane by randomly drawing three points from the point clouds Dt and constructing corresponding plane.The resulting candidate planes are tested against all points in Dt to determine how many of the points are well approximated by the candidate planes(called the score of the plane).After a given number of trials, the plane which approximates the most points is extracted and is defined as ground plane denoted by Pg.
Typical values for the parameters of RANSAC are: 10,000 iterations and ε = 3r.Where ε is largest distance allowed between the inliers and plane.Fig. 5(b) shows a visualization of ground plane detection results on the data of Fig. 5(a).Based on the ground plane detected, our algorithm removes ground plane from the Dt.Specifically, we remove all the inliers denoted by P p g .Inliers here are points with distance from the ground plane Pg less than ε = 3r.Once this operation has been performed, the different structures are no longer connected through the floor, so they could be clustered by labeling neighboring 3D points on the basis of their Euclidean distance.Thus, we cluster points of Dt−P p g using region growing to obtain a set of clusters, as shown in Fig. 5(c).
Each of these clusters is grown from a point pi, an unclustered point, in Dt − P p g .A new point pj is added to the cluster if it is the single nearest neighbor of a point pm which is already in the cluster.In order to avoid the clusters formed due to clutter, clusters with less than a minimum number of points (500) are discarded.Then, the largest cluster in C, in terms of number of points, is denoted by C * .Without loss of generality, we assume that the input point clouds mainly consist of one pig standing on a planar ground plane with possible parts of other facilities.Thus we can infer that the largest cluster C * contains all points lying on the livestock.Fig. 5d shows a visualization of pig segmentation results on the data of Fig. 5a.

Automated correct posture recognition using geometric features
Based on the segmentation results from the previous stage, we manage to construct the geometric elements mentioned in Sec.4.1 by exploring geometric constraints of four legs with ground plane so as to recognize the pigs with correct posture.

Geometric elements extraction
In order to construct the rectangle formed by four hooves, we begin by computing the intersection between one horizontal plane above the ground plane and the pig point cloud C * .Specifically, we obtain the upwards direction by using the relative positions of ground plane and pig as follows.We define one vector vu to represent upwards direction using following equation: where v pig is the center of mass of C * with N points, v gd is the center of mass of P p g with M points, both C * and P p g are point cloud of pig and ground plane from Sec.4.2 respectively.Let ng denote the normal vector of ground plane P p g .In general, there is no mathematical way to make sure that the normal vector of ground plane points upwards.Thus we reverse the direction of the vector ng only if ng • nu < 0.
Given the vector, we start from determining one horizontal plane P h by moving ground plane Pg along the vector ng a distance of d in the postive direction.Here d is a parameter which has to be set big enough to avoid the errors induced by ground plane estimation, is set to 5r for the point clouds data from Xtion pro in our case.Then the points set in C * lying on the plane P h is obtained by determining the set of points with distance from the plane P h less than f , is denoted by H.Note that f is another parameter which determines the thickness of our plane in the discrete sense.Thus the different hooves are on longer connected through the body, so they could be clustered by labeling neighboring 3D points on the basis of their Euclidean distance.Thus, we cluster points of H using same procedure performed in Sec.4.2, as shown in Fig. 6.
Luckily, pigs have four hooves, so we can infer that each of these clusters is corresponding with one hoove.Subsequently, we can easily get one quadrilateral by computing the convex hull of a four mass center points of those clusters.Fig. 6 shows a visualization of the quadrilateral extracted result on the data of Fig. 4.

Correct posture recognition
Based on the quadrilateral from previous stage, we can measure how much the C * agree with the first item in Sec.4.1 by using the accumulated difference between each angle of the quadrilateral and right angle as following equation.
Where S1 is measurement defined for determining how good is current posture according to the first item in Sec.4.1, ai is radians of one of interior angles of quadrilateral.Obviously, the smaller the measurement S1, the better the posture.Eventually, we can compute the S1 for each frame in D so as to classify it using a threshold T .

BODY MEASUREMENTS APPLICATION
Since the implemented 3D scanning system and the correct posture recognition algorithm can obtain the digitized live pigs with correct posture, it is applicable to body measurements of multiple livestock species with four legs.To demonstrate the potential of the algorithm, we have implemented a livestock body measurement software framework that integrates the proposed approach as one critical step.A screen shot of the software as a collage of some of its features is shown in Fig. 7.For more details about the measurement software, we refer readers to our published paper (Guo et al., 2017).

RESULTS AND DISCUSSION
We present results of our 3D scanning system, correct posture detection method and body measurement application respectively.The dataset we validate our correct posture recognition method,   Figure 7. Selected features of the developed livestock body measurement software (Guo et al., 2017).
body measurement software on is point clouds sequences of live pigs.The heights of these animals were between 52 and 66 centimeters(cm), while their lengths varied from 80.5 to 104 cm.All datasets are processed with constant parameter values.In detail, the resolution of the point cloud r is set equal to 0.04 cm, the octree leaf size for downsampling dr is set equal to 2r, the distance between ground plane and virtual plane d is 5r, the thickness of virtual plane f is 10r, while the parameters for RANSAC are: 10,000 iterations and ε = 3r.Where ε is largest distance allowed between the inliers and plane.Scanning speed results are reported on an Intel Core i7-2670QM CPU at 2.20 GHZ.
Figure 8.The final setup of our 3D scaning system for live pigs.
Results for 3D scanning system.The first step we validate our study is scanning live pigs under farming circumstances.Specifically, ten Landrace pigs with long bodies, short hair ranging in age from 130 to 220 days were scanned by using our 3D scanning system at the ShangDong WeiHai swine-breeding center of DA BEI NONG GROUP.Fig. 8 shows our portable scanning setups, while Fig. 9 shows visualization of point clouds sequences for one pig.Our 3D scanning system is capable of scanning the target at 10 frames per second(FPS).Although it is enough for scanning the live pigs, scanning speed can be improved to 15 FPS by using IO efficient saving device, solid state drive, etc.The accuracy of registration of two point clouds from two depth cameras is dependent on correspondences chosen manually.Another error source of scanning result is the Xtion Pro live sensor.
Basically speaking, the random error of depth measurement is proportional to the square distance from the sensor to the object.Which is well analyzed in (Khoshelham, 2011).Thus we only present the accuracy evaluation of scanning system in terms of our body measurement application in Sec.6.
Results for correct posture detection Here we present correct posture detection results on the point cloud sequences of 10 pigs.different posture.However, our method failed to acquire correct quadrilateral formed by hooves due to missing legs caused by occlusion.
Results for body measurement application The final experiment was carried out to validate our body measurement application.Specifically, each subject of 10 live pigs was manually measured using the Lydtin stick in order to assess heights, widths and lengths while the circumference was evaluated through a tape meter.We performed the corresponding body measurements on frames detected with correct posture of these live pigs by using the software.Figure 7 illustrates the measuring interface of the software.1. Comparisons between the mean value of live pigs body measurements measured manually and using our software.Ave: average percent errors for different traits on the 10 pigs.SD: standard deviation for corresponding percent errors.BL: body length.WW: withers width.WH: withers height.HW: hip width.HH: hip height.

CONCLUSIONS
In order to measure body traits of pigs, we have implemented a 3D scanning system for live pigs under farming circumstances that is capable of save the point cloud sequences at speed of more than 10 FPS.Additionally, we have presented a novel approach for detecting frames with correct posture from the point cloud sequences that is fully automatic and is able to recognize the frames with correct posture.Our approach can be pre-processing step for body measurement of pigs.We have shown results on ten pigs including comparisons with current measuring method that demonstrate that our method and application are competitive even though it makes less restrictive assumptions about the scaning data.
There are two limitations of our method.The first is that if the scanning subject is larger, such as cow, etc, our scanning system cannot work properly.This can be addressed by using more depth cameras, as in (Macknojia et al., 2013).The second limitation is that our current implementation of body measurement cannot handle scanning data that appear missing legs due to occlusion.
Missing legs do not allow the algorithm to construct correct S1.The implementation will be augmented to handle this case in our future work by taking the second assumption in Sec.4.1 into consideration when computing S1.Additionally, we will have the complete calibration procedure fully automatic without manual selection of point correspondences in the future.
(a) Users choose multiple corresponding points.(b) Calibration results.

Figure 2 .
Figure 2.An example of calibration session by using our calibration tool.

Figure 3 .
Figure 3. Whole pipeline for scanning live pig

Figure 4 .
Figure 4.One example of qualified scanning frame with correct posture.
(a) Input point clouds.(b) Ground plane detection.(c) Clustering points of Dt − P p g using region growing.(d) Pig segmentation.

Figure 6 .
Figure 6.Geometric elements extraction results on the data of Fig.4.Note that semi-transparent grey plane represents the horizontal plane P h , four colored clusters indicate {h1, • • • , h4} while four small semi-blue spheres represent the mass centers.Quadrilateral with cyan color is final output of geometric element extraction.

Fig. 6
Figure 9. Visualisation of point cloud sequences capturing in pig farm.