COAXIALITY CALCULATION METHOD FOR DROPPING OPERATION OF LUNAR SURFACE SAMPLING MISSION BASED ON MONOCULAR VISION USING ELLIPSE AND LINE FEATURES

In the lunar surface sampling mission, the drop operation that transferring the packaging container into the sample return container is a vital step. In this paper, for supporting the drop operation, we proposed a method to measure the coaxiality between the two containers based on the monocular vision pose measurement using ellipse and line features. The method first extracts ellipse and line features in the 2D image to calculate the initial parameters. Then, the initial parameters and the point set are used to fit the curve by the cylinder perspective contours fitting method, where the constraints between ellipse and line features are used to improve the robustness to varying noises. Next, given the fitting ellipse and camera parameters, the pose of the packaging container is solved by using circlebased method. Finally, the coaxiality of the two containers is calculated based on the relative pose between them. Simulation experiments were carried out and the results show the effectiveness and reliability of the proposed method.


INTRODUCTION
The lunar mission in the third phase of the Chinese Lunar Exploration Program will perform the tasks of landing, sampling and returning to the earth (Ye et al., 2014). Transferring precisely the packaging container into the sample return container is the vital step in the whole sampling operation. To ensure that the packaging container is precisely dropped into the sample return container, the coaxiality between the two containers is used as the criterion for judging whether the packaging container can be transferred into the sample return container.
Coaxiality measurement methods can be generally divided into three types: mechanical collimation method (Zhang et al., 1997), optical collimation method and laser collimation method (Li et al., 1998). These traditional measurement methods are unsuitable for deep space environment, because they have high requirements for equipment and have lower automation. With the development of optical sensors and image processing technologies, measurement methods based on computer vision have become an important means of coaxiality measurement. It has the characteristics of non-contact, high precision and high degree of automation. Applying computer vision technology to the coaxiality measurement system, the detection process is completely changed from manual measurement to automatic measurement, which is very suitable for the lunar surface sampling system. In addition, most of the existing methods of measuring coaxiality using computer vision techniques are carried out on a two-dimensional image space in pixels (Zou et al., 2006;Wang et al., 2015). In this paper, the computer vision technology is used to calculate the three-dimensional relative pose between space objects, and then the coaxiality is solved on the three-dimensional object space. * Corresponding author.
In the mission, the monitoring camera fixed on the upper surface of the ascent vehicle will take pictures during the process of pot dropping. Thus, the monocular vision pose measurement can be used to measure the pose between the packaging container and the sample return container (Wang et al., 2019). At present, monocular vision pose measurement methods are mostly based on point features David et al., 2004;Ning et al., 2018). Given a set of 3D points (e.g., marks and corners) and their corresponding 2D points, the pose of the target object can be solved. This is also called the PNP problem (Inaba et al., 2000). The point-feature-based method needs to consider the complex correspondence between spatial points and image points, which is a classic problem in computer vision. In addition, the pointfeature-based method might fail on the pose estimation of nonplanar objects, since few point features can be extracted from the image (Liu et al., 2014). The packaging container carried by lunar probe has no artificial target. During the dropping process, image quality may be poor with noise points or being affected by illumination conditions, it may be very hard to extract point features. Compared with point features, circle features are more abundant and stable in the image, and they have the advantages of easy recognition, anti-interference and anti-occlusion, but no complex matching problem. Therefore, the circle features of the cylindrical container are chosen to estimate the pose.
In general, the shape of a circle projected on the image plane after perspective projection is an ellipse. Huang et al. (1996) used the projected ellipse on the bottom of a cylinder to estimate the pose of the cylinder. The accuracy of elliptical contour recognition will directly affect the accuracy of pose estimation in cylindrical space. Ying et al. (2016) developed a constrained implicit equation to enhance the robustness and precision of perspective contour extraction which consists of ellipse and line features for cylinders. In order to improve the accuracy of ellipse recognition and pose estimation, this method is used to obtain the equation of image ellipse and derive the pose of the circle plane with respect to the camera frame. Based on the results of pose estimation, we can determine the relative pose of the containers and calculate their coaxiality in the three-dimensional object space, which can support the drop operation in the lunar surface sampling mission.
The rest of this paper is organized as follows: Section 2 summarizes the definition of coordinate systems; Section 3 specifies the proposed method; Result of simulation experiments are shown in Section 4; Finally, conclusions are given in Section 5.

COORDINATE SYSTEM DEFINITION
First, we establish the coordinate systems used in this paper. As shown in Figure 1, the coordinate frames are defined as follows: the packaging container coordinate system O-XYZ, the sample return container coordinate system Ob-XbYbZb and the camera coordinate system Oc-XcYcZc.
The origin O of the packaging container coordinate frame is located at the center of the bottom circle of the container. The Zaxis aligns with the central axis of the container. The O-XY plane is perpendicular to the Z-axis, and the X-axis points to the projection of the optical center of the camera on the O-XY plane. The Y-axis is determined by the right-hand rule according to the X-axis and Z-axis.
The origin Ob of the sample return container coordinate frame is located at the center of the top circle of the container. The Zb-axis aligns with the central axis of the container. The Ob-XbYb plane is perpendicular to the Zb-axis, and the Xb-axis points to the projection of the optical center of the camera on the Ob-XbYb plane. The Yb-axis is determined by the right-hand rule according to the Xb-axis and Zb-axis.
The origin Oc of the monitoring camera coordinate frame is located at the optical center of the camera. The Zc-axis is forward along the optical axis. The Oc-XcYc plane is perpendicular to the optical axis, and the Xc-axis and Yc-axis are parallel to the columns and rows of the image coordinate system, respectively.
The image coordinate frame is a 2D frame. Its origin is located at the upper-left corner of the image. The horizontal axis u and vertical axis v are the number of columns and rows in the image array.

METHODOLOGY
As shown in Figure 1, the cylinder perspective contours are composed of an ellipse and a pair of straight lines formed by the projection of its bottom surface and side. The flowchart of the calculation of coaxiality is illustrated in Figure 2. We first extract ellipse and lines in 2D images, and use their constraints to improve the accuracy of ellipse fitting. Based on the ellipse feature, the relative pose of the packaging container and the camera is solved. Finally, the coaxiality between the two containers is calculated.

Feature Extraction
The line and ellipse extraction process is based on the edge binary map of the image, which is obtained by Canny edge detector (Luckett, 2018). Hough transform is applied to detect the side straight line group and the bottom ellipse of the cylinder perspective contour quickly and roughly. Next, initial values of the parameters are calculated for the subsequent curve fitting.
The equation of an ellipse is denoted by: The equation of a line is denoted by:

Pose Estimation
The equation of image ellipse and the method proposed by Huang et al. (1996) are used to find the normal vector and the coordinates of the centre of the circle plane with respect to the camera frame. Figure 3 shows the elliptical projection of space circle on the image plane. The backward projection of the ellipse of the image is the elliptical cone surface. In order to determine the pose of the space circle, it is necessary to find a plane in space that intersects the cone in circle with the same radius R as the cylinder bottom surface. Since the elliptical cone equation is too complex to calculate in the camera frame, we need to find a new reference frame in which the cone has the standard form. The pose parameters of the space circle can be decomposed from the plane equation, then transformed back to the camera frame. First, the camera model is denoted by: where 0 is the focal length of the camera ( , , ) is the coordinate in the camera frame Substituting Equation (7) into Equation (1) and rearranging, we can get the equation of the cone: where = 0 2 , = 0 2 , = 0 2 , = 0 , = 0 , = .
Equation (8) Next, we define a new frame of reference ′ ′ ′ whose origin is the same as that of the camera frame. Thus, we can guarantee that the transform matrix between the two frames is a pure rotational matrix. Let represents the rotation matrix: Substituting Equation (10) into Equation (9), that is In order to express the elliptical cone surface in a standard form, is required to be a diagonalizing matrix for : Thus, the standard elliptic cone equation is: Based on the elliptic cone surface equation, the center and surface normal of the circle can be obtained as: The above solution is relative to the ′ ′ ′ frame. And the transform matrix will bring them back to the camera frame:

Coaxiality Calculation
The coaxiality is defined as the maximum distance between the top/bottom center of the packaging container and the axis of the sample return container.

Figure 4. Schematic diagram of coaxiality
As shown in Figure 4, the coaxiality is the maximum value between 1 and 2 . After solving by the above steps, we have obtained the center 1 ( 1 , 1 , 1 ) and the normal ( 1 , 1 , 1 ) of the bottom surface of the packaging container. According to the container's dimensions, we can calculate the top center 2 ( 2 , 2 , 2 ) of the packaging container in the camera frame.
( 2 , 2 , 2 ) = ( 1 , 1 , 1 ) + ℎ × ( 1 , 1 , 1 ) where ℎ is the height of the packaging container As the relative position between the sample return container and the camera is fixed and known, we can obtain the center ( 1 , 1 , 1 ) = ( 1 , 1 , 1 ) + ( 2 , 2 , 2 ) = ( 2 , 2 , 2 ) + where represents the rotation matrix between Ob-XbYbZb and Oc-XcYcZc represents the translation matrix between Ob-XbYbZb and Oc-XcYcZc We use the axis of the sample return container as the reference axis, and its direction vector = (0,1,0). The intersection point of the reference axis and the XOZ plane is 0 = (0,0,0). Thus, the distances d1, d2 can be determined, d1 represents the distance between and the top center 1 , d2 is the distance between and the bottom center 2 of the packaging container, Finally, the coaxiality between the two containers can be obtained:

EXPERIMENT
In this section, simulation experiments were performed to verify the method presented in this paper. In the experiment, a cylinder with known dimensions is given to simulate the packaging container. The radius and height of the cylinder are 50mm and 100mm. A set of camera parameters were also given to simulate the camera model. The simulation intrinsic matrix of the camera is: First, we project the cylinder onto the 2D image plane by the pose parameters based on the camera model. Figure 5 shows the projection result on the image plane.

Figure 5. Simulation projection result
We implemented the Hough transform method to detect the projected lines and ellipse. Simultaneously, the theoretical values of the curve parameters are calculated. And the data of point set was obtained based on the theoretical value of the curve parameters. We added Gaussian white noise points to the point set for simulating the sampling error of image points.
In order to verify the performance of the curve fitting method, we selected a set of parameters that deviate from the theoretical values as the initial estimated parameters. We first calculated the square of the approximate shortest distance from each point in the point set to the curve, then we selected the points whose are smaller than the threshold as the fitting points. Finally, we fitted the curve using the method in Section 3.2. The curve fitting result shows in Figure 6. Figure 6. Curve fitting result Figure 6 shows the initial curve and the fitted curve. The initial curve has certain distances from the theoretical curve points. In contrast, the fitted curve significantly reduces the distances from the theoretical curve points and improves the accuracy of the curve parameters.
After the fitting process, we calculated the pose of the cylinder using the pose determination method in Section 3.3. The pose consists of the center coordinates 1 ( 1 , 1 , 1 ) and the normal vector ( 1 , 1 , 1 ) of the circle plane. In order to show the cylinder's pose distinctly, we converted the normal vector to spatial angles. Since the rotation of the space circle around its normal vector has no effect on the projected ellipse formed in the image due to the symmetry of the circle, we can only get two spatial attitude angles measured by the circle, the pitch angle and the roll angle : Based on the 3D pose and dimensions (l=100mm) of the cylinders, the coaxiality was calculated using the method in Section 3.4. Table 2 shows the initial estimates and the final results of the center 1 ( 1 , 1 , 1 ), the pitch angle , the roll angle , the coaxiality and their respective absolute errors relative to the theoretical values.

CONCLUSION
This paper presented a method for measuring the coaxiality between the packaging container and the sample return container based on the monocular vision pose measurement using ellipse and line features. This method can improve the accuracy of the results when the ellipse and line contours with certain errors extracted by conventional methods. Simulation experiments were carried out and the results show that the method is effective and reliable. The coaxiality calculated by this method can be employed and better support the lunar surface sampling mission.