Robust features for Leg Detection in 2 D Laser Range Data

People detection in 2D laser range data is widely used in many application, such as robotics, smart cities or regions, and intelligent driving. For most current methods on people detection based on a single laser range finder are actually leg detectors as the sensor are always established below the knee height. Current state-of-the-art methods share similar steps including segmentation, feature extraction and a machine learning-based classification, but use different features which have good performance on their own experimental data. For researchers, it is important and desirable to know which features are more robust. In this paper, taking advantage of the fact that effective features can be selected by AdaBoost and assembled into a strong classifier, a set of features presented in state-of-the-art methods is combined with a set of features presented by us to train a leg detector by the AdaBoost algorithm. This detector is assembling by effective features and can classify segments into leg and non-leg. Three open source data sets including simple and complex scenarios are used for the experiments to test the features and extracted the important ones. To reduce the effect of segmentation on the final results, three segmentation methods are simultaneously used for experiments and analysis to ensure the reliability and credibility of our conclusion. Finally, 10 robust features for leg detection in 2D laser range data are presented based on the results.


INTRODUCTION
Recently, owing to the laser range finder's advantages of high accuracy, wide field of view, high scanning rates, minimal sensitivity to illumination changes and weather condition, availability of collaboration and lack of privacy issues, it is widely used in scene perception and has attracted increasing attentions (Armeni et al., 2017;Espinace et al., 2010;Weinmann et al., 2014).Hereinto, people (or pedestrians) detection is one of the hottest topics and used in various fields of applications, such as surveillance systems (Hou and Pang, 2011;Andriluka et al., 2008), flow analysis (Zhao et al., 2009), and driver assistance or autonomous driving (Geronimo et al., 2010;Althoff et al., 2009;Aufrè re et al., 2003).What's more, it is indispensable for service robots to detect, track and react to humans in their vicinity which is a necessity for Human-Robot Interaction (HRI).For most current methods on people detection based on a single laser range finder (LRF) are actually leg detectors as the sensor are always establish below the knee height, for example, on a mobile robot (Aguirre et al., 2014;Chung et al., 2012).
Existing methods have similar steps.Firstly, segmentation is carried out to decompose laser range data into segments.Segmentation plays a key role in object detection and significantly affects the performance.However, it has never been the research focus for people detection in 2D laser range data.Current methods are almost totally fall in the breakpoint detection (BD) method.This kind of method compares the distances between consecutive points with the threshold and tries to find the breakpoint.This threshold is generally a user-defined value (Weinrich et al., 2014;Mozos et al., 2010;Spinello and Siegwart, 2008;Arras et al., 2007) or determined by a geometric rule (Li. et al., 2018;Borges and Aldon, 2004;Carballo et al., 2011;Kim et al., 2016).
Then, the features of each segment are calculated and used to classify the segment into leg and non-leg.These features are the most crucial basis to differentiate legs from others.In recent years, an increasing number of features has been constantly put forward.Firstly, motion features are used to detect pedestrians, but human is not always moving.Then geometric features are designed and extracted for detecting circular shapes (Xavier et al., 2005), which is a better choice for people detection, especially for standing or sitting people.At first, threshold ranges of features are manually determined and adjusted by researchers to detect people.This manual work is time consuming which motivates the application of supervised learning.Arras et al. (2007) firstly proposed an approach that utilizes a supervised learning technique to create a classifier that facilitates the detection of people in laser range data.They defined 14 features and firstly applied AdaBoost to train a strong classifier to detect leg segments and demonstrated the good performance of their geometric feature set.Then, Spinello and Siegwart (2008) presented multidimensional features and shape descriptors that describe geometrical and statistical properties of the segment to detect legs in 2D range data and obtained desirable detection result.Carballo et al. (2011) studied laser intensity and textiles, introduced a group of new intensity-based detection features, and proposed a method for segment separation based on laser intensity.They demonstrated that their proposed intensity-based features improved the detection result.However, the intensity information could not be obtained for a portion of laser scanners.Weinrich et al. (2014) presented robust features for detecting people without walking aids, people with walkers, people in wheelchairs, and people with crutches in a rehabilitation center.However, their features are calculated with manually defined parameters and the default parameters may not be suitable for different kinds of environments.Lately, Li. et al. (2018) proposed a multi-type features method for leg detection in 2D laser range data.Three types of features, including relative distance statistical features, spatial relationship features and nearest neighbour features, are introduced and combined with some classic geometric features to train a strong classifier by the AdaBoost algorithm.The experimental results show that their proposed features are robust and effective in detecting legs in their datasets.Dozens of features are defined and extracted in current methods, which all show good performance in their dataset.Actually, these methods share some same features.And even some different features with different computational procedure in their methods may belong to same types and have similar effects.Which features are more robust for people detection in 2D laser range data?It is a valuable question that many people are concerned about and interested in.This paper aims to answer this question.
In this paper, three open source data sets including simple and complex scenarios are used for the experiments to test the features and then extracted the robust ones.Firstly, these data are divided into segments by three segmentation methods to reduce the effect of segmentation on the final result.Then a set of features proposed in state-of-the-art methods and by us are presented.Features of each segment are calculated and used to train a strong classifier with the corresponding labels by AdaBoost.Finally, more robust features can be obtained from the importance of the individual feature weights in the final strong classifier.
The rest of the paper is organized as follows: Section 2 introduces the leg detection process including segmentation, feature extraction and classification by AdaBoost.Section 3 reports the experimental results and presents the robust features.Finally, conclusions and future work are presented in Section 4.

METHODOLOGY
The proposed method aims to detect better features for leg detection in 2D laser range data.The overall workflow of this approach is presented in the Figure 1.
Labeled 2D laser range data Figure 1.The overall workflow of our method.

Segmentation
For each scan frame, distances between consecutive points are compared with a threshold to find the breakpoint.This threshold is generally a user-defined certain value which is one of the simplest method (JD method).This threshold may be adaptive determined by a geometric rule which is called adaptive breakpoint detector (ABD method) and can be defined as: where   is the laser beam range of point   ∆ is the angular resolution of sensor λ is the worst case of incidence angle of the laser scan ray with respect to a line for which the scan points are still reliable  =  (  ) is the noise with a radial error term that prevents unnecessary separation.
(  ) denotes the size of the radial error at range   , which is defined according to the LRF's specifications as: In addition, to simplify the calculation, the range difference rather than the distance between two consecutive points can be adaptive adjusted and used to compare with the threshold (ARD method).Because only laser ranges are variable while the angle resolution is a fixed and small value for each scan frame.The range difference threshold between two consecutive points ( −1 ,   ) is defined as: where   is the laser beam range of point   ∆ is the angular resolution of sensor  =  ( −1 ) + (  ) also denotes the noise with a radial error term that prevents unnecessary breakpoints Thus, if the threshold is larger than the distance or range difference, assign   to the segment of  −1 , or assign   to a new segment.This process decomposes each scan frame into segments, each of which corresponds to an object.Those segments cannot meet the minimum size criterion (at least three points) will be removed and not used for further feature extraction and leg detection.

Feature extraction
In this section, we divided the major features used in the previous research into four types, and presented the definitions of them in the following.
(1) Geometric features including basic parameters that measure the size of segments, shape descriptors that measure the convexity and compactness of segments and simple shape fitting parameters that measure the difference between the segments and simple and regular shapes are presented in Table 1.(2) Statistical features including basic parameters that measure the point number size of segments; point-based and laser beam based statistical features can reflect the distribution pattern of points and laser beam ranges, respectively; and the relative distance statistical features are presented in Table 2.For each segment, the line between the start point and the end point is regarded as the baseline, and the distance to the baseline is calculated for every other point.If the laser beam of a scan point intersects the baseline, then the relative distance () of this point is equal to the distance; otherwise, it is equal to the opposite of the distance value.

Types Features
Basic parameters  The distance between a person's legs falls within a limited range; thus, a finite length  can be used to restrict the search scope for each segment (e.g.,  = 1.0m in our experiments).When no additional segment is identified within the search range, the classification of the segment is fully dependent on its own features and the segment is regard as its own nearest neighbor.Thus, all the features above-mentioned can be clearly and simply expressed by the following formula: where   is a feature of segment   (a total of )  _ is one of the nearest neighbour features (a total of ,  ≤ ) Dozens of features for leg detection in 2D laser range data are defined and presented above.Although some of them may belong to same types or have similar effects, all of them are further used to train a strong classifier by AdaBoost in which effective and robust features can be extracted.

Best Features extraction
The AdaBoost algorithm is introduced to train a strong classifier T by using the proposed features for leg detection and to extracted best features.) where ) is the corresponding weight of ℎ  () In this paper, the features in F and the label of a segment forms an example where the label of a leg segment is +1 and the nonleg is −1.A set of training examples is formed and inputted to train a strong classifier after  iterations.The final strong classifier is a weighted combination of the best  weak classifiers.The best features for leg detection can be identified based on the importance of the individual feature weights in the final strong classifier.

Introduction to data sets and detectors
Three open source data sets are adopted for our experiments which are publicly available on the Internet: People2d 1 : the data set from (Spinello and Siegwart, 2008), which was recorded by a static laser scanner.Therefore, there is minimal variance in the background, and the backgrounds of the training and testing data are the same.
HOME & REHA 2 : the data sets from (Weinrich et al., 2014), which were recorded by a laser scanner on a mobile robot.Therefore, the background of the data set is diversely structured, and different rooms are used for the training and testing data sets.
The points (laser beams) in the original data sets are manually labelled as leg or non-leg.Additional detailed information about the three data sets is listed in Table 4.
All the features mentioned above are combined to generate a detector to compare with the state-of-arts detectors in Table 5 and find out the robust features from them.
The strong classifier used in this paper that comprises 50 weak classifiers, and each weak classifier is a stump.

Segmentation Results
Three segmentation methods are simultaneously used for experiments and analysis.The datasets used for training or testing are separated by the original data collector and nothing changes have been made by us.
JD: the jump distance threshold between consecutive points is 0.1m.
Although the segmentation results significantly affect the classification performance.The leg detection results are decided by the features for distinguishing legs from other objects.In the following section, the features are used to train a strong classifier by AdaBoost and tried to distinguish legs from other segments.

Evaluation Indicator
In this experiments, the evaluation is based on the segments rather than the combination people to more realistically reflect the classification ability of each detector.A leg segment that is accurately classified is counted as a true positive (TP), whereas a leg segment that is not correctly identified is a false negative (FN).
A non-leg segment that is misclassified is counted as a false positive (FP), whereas a non-leg segment that is accurately classified is a true negative (TN).Furthermore, additional indicators are introduced to evaluate the classification results and to compare the detectors: precision (P), recall (R) and F1 score.

Classification result
The classification results of the people2d, HOME and REHA data sets are respectively presented in Table 7, Table 8 and Table 9.
Since data set people2d is relatively simple, there are no much difference between detectors and all the detectors have a high precision, recall and high F1 score for leg detection which is shown in Table 7.Compared with the other detectors, the proposed detector ALL outperform others.
For dataset HOME and REHA which are more complicated that a part of legs are partially occluded legs and merged legs, features used in detectors of Arras, Spinello and Carballo can classify the leg segments correctly but cannot well detect leg segments from other segments.However, the detector ALL and DALIN still keep the good performance in these complicated datasets as shown in Table 8 and Table 9.It is because that there exist more robust features in detector ALL and DALIN.
The classification results showed that the segmentation methods have a certain effect on the results, but the results are more decided by the features used in the detectors.Although the different segmentation methods are used, the detector with robust features have maintained better performances.
In total, the detector ALL outperform others in the three datasets no matter what segmentation method is used because all the presented features are used in it.Thus the best features for leg detection can be extracted from the strong classifier of detector Although the feature  occupy the first place in the dataset people2d and also appear in other dataset, it was not been regarded as the robust feature.Because this feature that describe the distance between the segment and the laser range finder is not a truly effective and robust feature.Its good outstanding performance probably because of the limitation of the range of sensor or the ranges of manually labelling.

CONCLUSION
This paper analysed features for leg detection in 2D laser range data proposed in current state-of-the-art research and divided them into four feature types.These features are combined with a set of features presented by us to generate a feature space.This feature space including 70 features in total and fall into all of the four types.These features are used to train a strong classifier by the AdaBoost algorithm.Then more robust and effective features can be obtained from the importance of the individual feature weights in the final strong classifier.Three open source data sets including simple and complex scenarios are used for the experiments.In addition, three segmentation methods are also simultaneously used for experiments to reduce the effect of segmentation on the final result.Finally, ten robust and effective features are extracted from the experiment result.
These ten features are not the features with top 10 importance weights but the better features except for the influence of possible human factors.They show good and stable performance no matter what segmentation method or dataset is used in our experiments.Thus they can be considered as robust and effective features for leg detection in 2D laser range data.The proposed method for the extraction of robust features for leg detection can be used for improving the classifiers and expanding to other areas.More features with better discriminability and generalization ability will be explored in the future to further improve the performance for leg detection and the extraction of other object in both 2D laser range data and 3D laser scanning data.
Euclidian distance between the first and last point of a segment 02.Radius(): from the best fitting circle 03.Boundary length(): sum of the distances between two adjacent points in the segment Shape descriptors 04.Mean curvature(): mean for each of the curvatures   computed from the triangle formed by every three adjacent points 05.Mean angular(  ): mean value of the angles formed by every three adjacent points 06.Kurtosis(): to the center of gravity of a segment 07.Size ratio(  ): ratio of the sides of the bounding-box enclosing the segment Simple shape 08.Linearity(): variance of the residuals to the best fitting line fitting parameters 09.Circularity(): variance of the residuals to the best fitting circle 10.Ellipticality(): variance of the residuals to the best fit ellipse 11.Linearity normed(  ): linearity value divided by the number of points 12. Circularity normed(): circularity value divided by the number of points 13.Ellipticality normed(  ): ellipticality value divided by the number of point 14. Number of points()15.Normalized number of points ( ̂): the ratio of the actual number of points and the maximum expected number of points at a given range,  ̂= 2 • (/2)/  , where  is the range to the segment center,  is the angular resolution of the sensor and   is the maximum segment width Pointbased 16.Standard deviation(  SD ):    18. Boundary regularity(  ): standard deviation of the boundary length Laser beambased 19.Maximum length of laser beams:  max =  max −  average 20.Minimum length of laser beams:  min =  min −  average 21.Maximum difference between laser beams:  diff 22.Standard deviation of laser beam ranges:  SD Relative distance 23.Maximum relative distance ( max ).24.Minimum relative distance ( min ).25.Average relative distance ( ave ). 26.SD relative distance ( SD ).27.Average relative distance of the first half ( ave1 ).28.Average relative distance of the second half ( ave2 ).

Table 1 .
Major geometric features

Table 2 .
Major statistical features(3) Spatial relationship features including point-based and laser beam based spatial relationship features that describe the spatial relation between a segment and its neighbours are presented in Table3.Preceding range difference (  ): the difference between the first laser beam range of the current segment   and the last laser beam range of  −1 .34. Succeeding range difference (  ): the difference between the last laser beam range of the current segment   and the first laser beam range of  +1 .
35.Sum of distances (  ): the sum of the absolute values of the  and .Table3.Major spatial relationship features (4) Nearest neighbour features tried to use the features of its nearest neighbour for further classification because a leg is likely to appear near another leg.These features are used to measure whether the nearest neighbour is a leg, which have effects on the probability of current segment being a leg.
(): χ → {+1, −1}) that have a low weighted error   .AdaBoost is adaptive in the sense that the weight distribution   is changed on each iteration and subsequent weak learners are tweaked in favor of those instances misclassified by previous classifiers.The final strong classifier (()) is composed of a weighted majority sum of the selected weak classifiers.
The input to the AdaBoost algorithm is a set of labelled training examples ( 1 ,  1 ),…, (  ,   ) where each   belongs to some domain or instance space χ and the label   ∈ {+1, −1}.The examples are initially weighted according to a distribution   .In a series of rounds, =1,…,, the AdaBoost algorithm selects the weak classifier (ℎ

Table 5 .
The state-of-the-art detectors

Table 6 .
Segmentation results of leg segments by different methods for each data set

Table 7 .
Classification results of leg segments by different methods for data set people2d

3.5 Best Features for Leg Detection Regarding
the different strong classifier trained, in Table10we present the top 10 features in each of the trained strong classifiers of detector ALL.The selection is based on the feature contribution to classification, measured as the unsigned sum of the feature weights across all weak classifiers (each strong classifier has 50 weak classifiers in our experiments).Table10shows that four types of features all played important roles in the detections.The statistical features are not as good as the others.Most of them are not appeared in Table10, which means only some of them are effective in leg detection.For example, the relative distance statistical features that can describe the concavity and convexity of the whole and part of segments show good performance.The spatial relationship features are dominant in all the strong classifiers which can be considered as the most effective and robust features for leg detection, especially the laser beam-based spatial relationship features  and  which are almost always in the top position.It may because that these features are more robust and stable even if the shape and size are changing.

Table 10 .
Top 10 features in each strong classifiers of detector ALL where geometric features are in black, statistical features are in blue, spatial relationship features are in red and the nearest features are in green.