TOPOGRAPHIC MAPPING WITH MANIPULATOR ARM CAMERA IN LUNAR SAMPLE RETURN MISSION

The topographic mapping of sampling areas, providing basic sampling environment information, is crucial in sample return mission. The fixed monitoring cameras were designed for mapping of sampling areas in fixed effective resolution. In order to perform more detailed topographic analysis of sampling areas, this paper proposed a topographic mapping method based on the sequential sample images captured with the movements of manipulator arm. The tie point matching results and the image exterior orientation parameters obtained from measurements of manipulator arm joints were employed to the weighted bundle adjustment based optimization for the accurate topographic mapping. The simulated images were adopted to validate the effectiveness and accuracy of the proposed method.


INTRODUCTION
In 1960s and 1970s, the U.S. and the former Soviet Union successfully implemented the lunar sample return missions and brought back hundreds of kilograms of lunar soil and rocks successfully (Allton et al., 2003;Bugos et al, 2013). These lunar samples have offered an fundemental basis for the studies of lunar sciences. China plans to launch Chang'e-5 lunar sample return mission in 2020 (Xinhua, 2020). As shown in Fig.1, the Chang'e-5 probe is composed of 4 modules: orbiter, returner, lander and ascender (Cai et al., 2019). The lander, carrying the ascender, will land in the northwest part of Oceanus Procellarum and take back at least 2 kilograms lunar samples (Gbtimes, 2017;Zhao et al., 2017;Xinhua, 2020). During the mission, the lander will accomplish soil and rock sampling with a 4-DOF manipulator arm in the sampling areas around the landing point. Then, the samples will be transferred to the ascender, and be brought to the orbiter after the processing of rendezvous and docking. After separating with orbiter, the returner will bring the samples back to Earth (Cai et al, 2019;Wang et al., 2019). Detailed topographic analysis of sampling areas is crucial to the effectiveness and safety of sampling operation. It provides not only the geometric information of potential sampling targets, but also the geomorphologic information of sampling areas for the guidance of target selection. In the mission, stereo vision based on lander monitor cameras will be applied for topographic mapping of the sampling areas. The binocular monitor cameras fixed on the lander captures the stereo images of sampling areas for the evaluation of sampling operation at variable intervals. However, the binocular cameras are designed for the regional monitoring of sampling areas and take images immovably in the sampling operation. The fixed resolution of sampling images may not satisfy the requirements of detailed analysis about lunar soil porosity, softness and so on. Considering the damage risk of the binocular monitor cameras, the 3D reconstruction method independent of lander monitor cameras, which could provide higher resolution topographic products, should be introduced to the mission.
For close sampling monitoring, two sampling cameras are installed beside the sampler in opposite direction at the end of manipulator arm. The sampling cameras follow the sampler movements driven by the manipulator arm and capture the sequential monocular images of sampling areas. The exterior orientation parameters (EOPs) of the sampling images can be calculated with measurement parameters of four arm joints (Wang et al., 2019). Therefore, it is feasible to use the sequence of sampling images for topographic mapping of the sampling areas based on structure from motion (SFM) (Schönberger et al., 2016). However, due to the flexibility of the manipulator arm and measurement errors of arm joints, the errors exist in the measured EOPs, which may decrease the accuracy of the topographic mapping results of the sampling images. This paper proposed an SFM based topographic mapping method with monocular sampling images. The uncertainty estimation of the measured EOPs was conducted for the optimized solution of 3D reconstruction. The simulated images, generated from the image simulation system for the sample return mission, were used for accuracy evaluation. Compared with the original DEM used for image simulation and the generated DEM with accurate EOPs, we analysed the feasibility and effectiveness of the proposed method.

METHODOLOGY
The workflow of the proposed method is shown in Fig. 2. Firstly, SIFT based feature matching (Lowe et al., 1999) in every pair of images is performed to obtain matched feature points. Secondly, relative orientation is applied for initial relative EOPs acquisition of the sampling images. In order to obtain the maps of sampling areas, the EOPs of each sampling image in the lander coordinate system, are calculated with the joint parameters of manipulator arm. In addition, the precision estimation of the calculated EOPs is conducted based on the position error model of the manipulator arm (Liu, 2014;Wang et al., 2019). Then, these EOPs are incorporated into the bundle adjustment (BA) based optimization solution as weighted observations. Finally, we use Semi Global Matching (SGM) (Hirschmüller, 2007) to obtain the topographic mapping results of the sampling area.

Sampling image matching
SIFT method has the ability to handle the matching between images involving changes of rotation, scaling and illumination. We used SIFT to extract the feature points having discriminative feature values. Then, the feature matching was applied to each pair of images, in order to find the corresponding relationships among the features of all images.
To obtain the even distribution of the matched points, the matched feature having highest matching value in local image area was chosen. In addition, the matching outliers were detected by the constraints formed by the fundamental matrix (Nisté r, 2004). Finally, these matched feature points were used as the tie points for construction of the image network.

Calculation of EOPs of the sampling images
The EOPs of the sampling images, captured by the sampling camera attached with the manipulator arm, can be calculated with the joint rotation parameters of the arm. The rotational ranges of four joints are list in Tab. 1. For the 4-DOF manipulator arm, the kinematic model was built with linkage coordinate system construction based on D-H method (Denavit, et al., 1955). The linkage coordinate system is shown in Fig. 3.

Fig. 3 The linkage coordinate system of manipulator arm
It is combined with one base and four joint coordinate systems, which are defined as X0-Y0-Z0, X1-Y1-Z1, X2-Y2-Z2, X3-Y3-Z3 and X4-Y4-Z4. The base and the fourth joint coordinate system were parallel to the lander and sampler coordinate systems respectively. Thus, by involving the parameters of joint rotation and arm part installation, the EOPs of sample camera can be obtained by the coordinate system transformation calculation of the manipulator arm, which is represented as: is the homogeneous transformation matrix (HTM) of sample camera in lander coordinate system; 0 is of arm base HTM relative to the lander coordinate system; 1 ( 1 ), 2 ( 2 ), 3 ( 3 ), 4 ( 4 ) are the relative HTM of the four arm joints with the rotation of angle 1 , 2 , 3 and 4 respectively; is the installation matrix of sample camera relative to the sampler. The can be calculated by given the measured rotation parameters 1 , , 3 and 4 .
Due to the joint measurement uncertainty of manipulator arm, the uncertainty of the calculated EOPs needs to be estimated for high accurate 3D reconstruction. For the i th joint in D-H model, the HTM is represented as: = ( , −1 ) • ( , −1 ) • ( , ) • ( , ) (2) where ( , −1 ) is the rotation matrix around axis −1 with joint angle ; ( , −1 ) is the translation matrix along −1 with joint length ; ( , ) is the translation matrix along with connecting rod length ; ( , ) is the rotation matrix around axis with twist angle . According the principles of complete differential, the HTM will change to ′ , which is represented as ′ = + ∇ .
(3) The ∇ is the error of the HTM based on differential transformation, which is calculated as: For the 4-DOF manipulator arm, the HTM of sample camera is represented as ′ = + ∇ The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2020, 2020 XXIV ISPRS Congress (2020 edition) where ∇ is the uncertainty brought by the measurement error of joint rotations, which is obtained by In addition, the connecting rods of the arm were made of flexible material. The deformation of the rod also brought about the motion estimation errors of the captured images. Before mission, the flexibility coefficients of the arm material were calibrated precisely. Thus, the EOPs errors of sample camera caused by arm flexibility can be eliminated through the flexibility error compensation.

Weighted BA based 3D reconstruction
The calculated EOPs of sampling images were taken as weighted exterior observations for geo-positioning of reconstructed model in lander coordinate system. The BA error model of the sampling imagery is represented as where 1 and are the observation correction vector of image tie points and calculated EOPs of sampling images respectively; and are the BA coefficient matrix of image EOPs and 3D tie points; and is the correction vector of image EOPs and 3D coordinates of tie points; is the coordinate residual vector of image tie points; and are the observation weight matrix of image points and image EOPs. Under the principles of least squares, the normal equation is built as By solving the correction vector [ ] iteratively, the optimized image EOPs and 3D coordinates of tie points are obtained.
From the sequential images, each image and its matched image are used to generate a single depth map based on SGM. Then, the final topographic mapping result is obtained by optimally fusing the multiple depth maps with image EOPs.

EXPERIMENTAL RESULTS
The topographic mapping accuracy of the proposed method should be investigated to evaluate the feasibility for sample return mission. The image simulation system of lunar sampling return mission was utilized for sampling image generation. Due to the complexity of manipulator arm EOPs estimation, multiple EOPs with different level precision were employed for topographic map generation. Then, the accuracy analysis was performed with original DEMs adopted in image simulation.

Sample image simulation
In the simulation system, we configured the EOPs and Interior orientation parameters (IOPs) of the simulated camera, and the scene data of terrain and texture. In order to simulate the illumination condition, the parameters of illumination model, such as solar elevation angle, azimuth angle and surface albedo, can also be adjusted according to the mission. The simulated images were generated from digital projection with the virtual sampling camera by using typical Digital Elevation Model (DEM) and Digital Orthophoto Map (DOM) of the lunar surface.
In this experiment, the seamless DOM, generated by sub-area block bundle adjustment with rational function model and whole area adjustment with thin plate spline model, was adopted to the simulation system. Meanwhile, we input the SLDEM 2015 as evaluation reference. In order for the simulated images to have enough details, the resolution of DEM and DOM was considered as 0.001m in the simulation computation. The field of view of the sampling camera used in the system was 46 degrees and its image size was 1024*1024 pixels.
Considering the typical resolution of topographic mapping by lander monitor cameras is 0.02m, we set sampling camera parameters so as to obtain simulated images with the ground sample distance of 0.002m and the overlapping ratio of adjacent images to be 60%. These images had potential capability to generate sampling maps with a grid spacing of 0.005-0.01m. To simplify the image acquisition mode, we chose 9 sampling camera images captured in equal interval with vertical downward direction for experimental analysis, which are shown in Fig. 2.

Topographic mapping results
SIFT method was employed to obtained the matched points among multiple simulated images as the tie points, which are shown in Fig.4. In the figure, the red lines and blue lines represent the matching outliers and correct matched points respectively.
(a) (b) Fig. 4 The matching results of simulated images. (a) shows the matching results between Image 1 and 4; (b) shows the matching results between Image 1 and 7.
Then, the image EOPs were input to the weighted BA based optimization processing to obtain the optimized EOPs. After multiple depth map reconstruction and fusion, the lunar surface DEM product was generated with a grid spacing of 0.005m and an image size of 400*400 pixels, which is shown in Fig. 5. Fig. 5 The DEM generated from the simulated images

Accuracy analysis
The uncertainties of the obtained EOPs of sample images brought about the errors of generated DEMs. In order to simulate the obtained EOPs of sample images, random errors were added by taking the uncertainty estimated in Section 2.2 as reference. In fact, when the sampling camera moved to the planned space position, the arm planning solution of the joint rotation configuration may not be unique, that brought about different EOPs errors. In the analysis, six configurations of EOPs uncertainties, set with different standard deviations of the random errors in EOPs, were employed to implement the topographic mapping and obtained the DEM products, which are list in Tab Fig.6 shows the distribution of the errors with configuration 6. Fig. 6 Height errors of the generated DEM with uncertainty configuration 6 The error distribution of the DEM without EOP errors involved is shown in Fig.7. The errors are much smaller than that in Fig.6, reflecting the best attainable accuracy without EOP errors involved.

Fig. 7 Height errors of a generated DEM without EOP errors involved
Based on above investigation, it can be concluded that the main error source of topographic mapping with sampling camera is the joint measurement errors of the manipulator arm. The accurate error modelling and compensation of the calculated EOPs with manipulator arm rotation measurements are esensstial for high-precision topographic mapping of lunar sampling areas.

CONCLUSION
This paper proposed a topographic mapping method with monocular sampling camera in lunar sample return mission. The EOPs uncertainties of the sample images, captured with the movement of the manipulator arm, were applied to the lunar surface topographic mapping. The simulated images were used for the effectiveness verification and accuracy evaluation of the proposed method. In the near future, indoor analogy experiments will be conducted to estimate the error model of the manipulator arm more accurately. The developed method can be employed to support the sampling operation in Chang'e-5 mission.