A GRADIENT-REGION CONSTRAINED LEVEL SET METHOD FOR AUTONOMOUS ROCK DETECTION FROM MARS ROVER IMAGE

: Rocks are one of the major Martian surface features and yield significant information about the relevant geology process and the life exploration. However, autonomous Martian rock detection is still a challenging task due to the appearance similar to the background, the view and illumination change. Therefore, this paper presents a gradient-region constrained level set method based on mars rover image for automatic Martian rock extraction. In our method, the evolution function of level set consists of the internal energy term for guaranteeing the deviation of the level set function from a signed distance function and the external energy term, where the gradient-based information is integrated with the locally adaptive region-based information, for robustly driving the motion of the zero-level set toward the object boundaries even in images with ununiform grey scale. The resulting evolution of the level set function is based on the minimisation of the overall energy functional using the standard gradient descent method. As a result, those detected Martian surface regions that are most likely to yield valuable scientific discoveries will be further analysed based on two-dimensional shape characterisation. To evaluate the performance of the proposed method, experiments were performed on mars rover image under various terrain and illumination conditions. Results demonstrate that the proposed method is robust and efficient for automatically detecting both small-scale and large-scale rocks on Martian surfaces.


INTRODUCTION
Currently, the new round of deep space exploration booms and the leading countries and organizations have initiated several deep space exploration missions in recent years, such as National Aeronautics and Space Administration (NASA)'s Lunar Reconnaissance Orbiter (LRO), Lunar Crater Observation and Sensing Satellite (LCROSS), Mars Global Surveyor, and Mars Odyssey, European Space Agency (ESA)'s Mars Express, China's Lunar Exploration Project, Japan's SELenological and Engineering Explorer (SELENE), which has provided a considerable number of reliable data and contributed to widespread research interests.For example, the acquired high-resolution satellite imageries from these orbiters were used for the high-resolution imaging and mineralogical mapping of the surface (Bibring et al., 2006), for radar sounding of the subsurface structure down to the permafrost (Picardi et al., 2005), to generate the gravity model (Smith et al., 1993), to inverse the heat flow activity over the Martian surface (Abramov and Kring, 2005), etc.The weather and climate conditions on Mars could be interpreted and analysed based on the observed atmospheric circulation and composition (McCleese et al., 2007).Spectrometers and thermal imagers could be used to detect evidence of past or present water and ice (Michalski et al., 2013), as well as study the planet's geology and radiation environment.
With the development of deep space exploration technology, the requirements that both lander and rover have the capability of independently and autonomously analysing information and selecting the valuable scientific data are increasing.To date, eagle 2, Phoenix, Spirit, Opportunity, Curiosity, Jade Rabbit and so on have been designed and landed on several terrestrial bodies, such as Moon or Mars, to perform exobiology and geochemistry research, and even sample return missions.In the future, deep space exploration missions will be able to collect more sensor data than can be transmitted to earth.Under the circumstances, autonomous in-situ scientific data analysis enables major increase in scientifically valuable data return without heavy downlink or remarkable time delay.
Mars, one of the planets in the solar system, exhibits similar characteristics to Earth and has always been a hotspot issue in the deep space exploration.For Mars exploration, it is highly desirable to analyze the imagery data for distinguishing objects (e.g.rocks, gravels or creatures) from background in images acquired from the Mars exploration rover (MER).Rocks are one of the major Martian surface features and their distribution and the physical properties can provide crucial information about planetary surface for many applications ranging from hazards avoidance, robotic route planning to further geological analysis.First, rocks are the ideal cross-site tie points for visionbased rover localization and navigation.Second, for many sites of scientific interests on Mars, the rock distribution is high enough to create a landing or rover failure probability.The position and distribution of rocks can guarantee the safe navigation missions and increase the accessible surface area.Third, according the type and distribution of the rocks, such as sedimentary or igneous, the topographic and physical characteristics (e.g., interior conditions, surface conditions or atmospheric conditions) of the environment around the landing site can be investigated.What the regions were like and the effect of climate or weathering can be deduced when the rocks were being formed and deposited.Consequently, independent and autonomous rock detection and analysis can help to guarantee the safe rover navigation and achieve scientific and engineering goals.
Rock segmentation procedure plays a crucial role for the success of Mars exploration rover mission and its scientific studies.In recent years, numerous studies about rock segmentation and detection derived from Mars images have been done.Gor et al. (2001) proposed an unsupervised hierarchical framework, where intensity information was used to detect small rocks and range information to detect large rocks, for autonomous rock detection on Martian terrain.Fox et al. (2002) segmented the rocks from the background based on intensity and height and classified the shape and other geologic characteristics of rocks from two-dimensional photographic images and three-dimensional stereographically produced data.Castano et al. (2004) constructed an image pyramid model for extracting different scale rocks from different levels, where at every level the edge detector and the edge walker were used to find closed shapes as rocks.Li et al. (2007) extracted and modelled rocks from three-dimensional ground points generated by stereo image matching as cross-site tie points to long-range autonomous Mars rover localization.Thompson and Castano (2007) 2013) adopted a mean-shift segmentation algorithm to generate a set of homogeneous objects and combined 3D point clouds derived from a pair of intensity images to extract both small and large rock candidates.Wang et al. (2015) investigated the imagery characteristics of Martian surface and model the interaction between two pixels of an image for differing foreground rocks from background information to keep rover safe navigation.Xiao et al. (2017) presented a new autonomous rock detection approach based on homogeneous region-level intensity information and spatial layout.Xiao et al. (2018) reconstructed background information using sparse representation and implemented a threshold segmentation on enhanced contrast map to precisely detect rocks for Mars rover.
Rocks in the Martian scene exhibit significant difference in morphology and the image intensity varies remarkably due to the illumination, which poses great challenges for automatically detecting these rocks.To address these challenges, in this paper, we develop a gradient-region constrained level set image segmentation method based on Mars rover image.Moreover, the shape characterisation of these extracted rocks is further analysed for studying the geological origins and history.The rest of this paper is organized as follows.Section 2 describes the proposed method in detail.Section 3 presents the experimental results and analysis for evaluating the proposed method.This paper concludes with a discussion of future research considerations in Section 4.

Principles of level set method
The level set method proposed by Osher and Sethian (1988)    satisfies the Eq.(1).Fig. 1 shows a curve given by the zero-level set of a level set function By defining a different energy term to represent information within the image domain, the evolving contour can change flexibly according to varying purposes.The existing level set methods can be generally divided into two categories: the edgebased models (Caselles et al., 1997) and the region-based models (Chan and Vese, 2001).
The geodesic active contour (GAC) model proposed by Caselles et al., (1997) is a typical edge-based model and solved by minimizing an energy functional in Eq. ( 4). where C is a curve parameterized by the arc-length s and ds denotes the arc-length element (Wang et al., 2014).Consequently, the gradient descent flow can be derived from the Euler-Lagrange of (4), as shown in Eq. ( 5).
where div denotes the divergence operator,  and v denotes constant coefficients and  is mainly used to stop the curve at object boundaries with high gradient values.For more details, please refer to (Caselles et al., 1997).
The region-based models take the region information into account.For example, the CV model (Chan and Vese, 2001) establishes the energy function in the frame of the Mumford-Shah functional (Mumford and Shah, 1985) for segmentation, . For more details, please refer to (Chan and Vese, 2001).

Gradient-region constrained level set model
As mentioned earlier, the conventional CV model only introduced the global region information into the evolution function without using the edge information, which results in the inaccurate object boundary detection especially for images with ununiform gray scales, while the typical GAC model often failed at the ambiguous rims.Therefore, we adopt a gradientregion constrained level set model image segmentation method integrating gradient information with region information for automatically extracting Martian surface rocks.
To introduce both the region information and gradient information into the evolution function, the energy function is constructed as follows In order to avoid the re-initialization during the evolution, the internal energy term ) ( int  E is used to penalize the deviation of the level set function from a signed distance function (Li et al., 2005), as defined in Eq. ( 9).
where  denotes a constant.
Then, the external energy term consists of two parts: the gradient-based energy term ) ( edge  E (Li et al., 2005) and the region-based energy term ) ( region  E (Li et al., 2007), as defined in Eq. ( 10) and Eq. ( 11).
where 1  and 2  denote two constants, only fit the intensity near x for alleviating the effect of ununiform gray.For more details about ) ( region 

E
, please refer to (Li et al., 2007).Finally, the standard gradient descent method is used to minimize the energy function Eq. ( 8).

Rock shape analysis
After the rocks over Martian surface are detected, their inherent shape characterization is valuable information for studying geologic origins and history (Blatt et al., 1980).Indeed, the shape of a rock is a complex property and is tough task to describe precisely.Referring to the basic concepts about classifying and categorizing the general appearance of microscopic particle grains in geological work (Dudek andTsotsos, 1997, Kwan et al., 1999), in this section, the ellipse fitting error and eccentricity of the fitting ellipse derived from modeling the rock by an ellipse within image spaces through a direct least-squares fitting method (Maini, 2008) are used as indicative measures of a rock's shape.
First, the ellipse fitting error between the fitting ellipse and the rock boundary is calculated as a measure of its relative roughness, indicating the sharpness of a rock's corner and the angularity of its edges.Its value ranges from 0 to positive infinity.the greater a rock's ellipse fitting error, the more angular its edge is.
Then, the eccentricity of the fitting ellipse is an important characterization of the rock and provides information with respect to the composition and history.In our method, let .This value ranges from 0 to 1.The smaller the value, the more circular the rock is.

EXPERIMENTATION AND ANALYSIS
To evaluate the performance of the proposed method, experiments were performed on images captured from Spirit Mars Rover Panoramic and Navigation Cameras along its traverse path under various terrain and illumination conditions.3 shows experimental results derived from the proposed method, suggesting that most rocks can be successfully extracted for further analysis.However, in some challenging regions, where rocks are stuck together, two or more rocks might be detected as one rock (as shown in Fig. 3(b)).In addition, the regions with large slope also reduce the detection performance (as shown in Fig. 3(d)).
Fig. 4 shows extracted rocks and their associated fitting ellipses, where fitting ellipse is represented by red dashed lines and rock boundary by yellow polygons.As shown in Fig. 5, the rocks are ranked for each measure.These measures provide an intuitive shape characterization of each rock.For instance, rock # 1, 7, 9, 15 has larger ellipse fitting errors, which indicated that they are more irregular than others.Furthermore, the eccentricity of rock #8 is minimal.We can conclude that compared with others, its appearance is more circular.

CONCLUSIONS
To independently and autonomously detect rocks on Martian surfaces, we develop a gradient-region constrained level set image segmentation method based on Mars rover image.In our method, the gradient-based information is integrated with the locally adaptive region-based information for robustly driving the motion of the zero-level set toward the object boundaries even in images with ununiform grey scale, which effectively alleviate the weak problem and increases the attraction of  the true rock edges to active contours.Meanwhile, the inherent geometric characterization of these extracted rocks is further analysed for giving valuable information with regard to both the geological analysis and scientific missions.Experiments were performed on Mars rover image under various terrain and illumination conditions.Results suggest that the proposed method is robust and efficient for automatically detecting both small-scale and large-scale rocks on Martian surfaces.Nevertheless, in some challenging areas where rocks are stuck together or covered with sands, the proposed method might not produce satisfactory extraction results.Enhancing the rock detection performance in these challenging areas will be our future focus.Additionally, constructing a framework for the rock classification task will be our focus.As a result, those detected Martian surface regions that are most likely to yield valuable scientific discoveries will be further explored using more scientific measurements.
compared the performance of seven existing rock detection algorithms on Mars Exploration Rover imagery, terrestrial images from analog environments, and synthetic images from a Mars terrain simulator.Dunlop et al. (2007) incorporated the local-scale, the object-scale and the scene-scale attributes into a learned rock appearance-based model for Martian rock detection and segmentation.Matthies et al. (2008) conducted stereo-based rock detection building on the surface plane fit approach for landing hazard detection.Song (2010) used texture-based image segmentation and edge-flow driven active contour for automated rock segmentation from Mars exploration rover imagery.Di et al. ( Fig. 2 shows the schematic diagram of level set method during evolution.At 0 = t , the initial position ) 0 , , ( ) 0 , , ( =  = = t y x d t y x 

G
denotes the Gaussian kernel with standard deviation  , and I denotes the image.The function g

a
and b denote the semi-major and semi-minor axes of the fitting ellipse, respectively.Supposing that Scene III (d) Scene IV Fig. 3 Experimental results derived from the proposed method.The detected rocks are marked with yellow polygons.

Fig.
Fig.3shows experimental results derived from the proposed method, suggesting that most rocks can be successfully extracted for further analysis.However, in some challenging regions, where rocks are stuck together, two or more rocks might be detected as one rock (as shown in Fig.3(b)).In addition, the regions with large slope also reduce the detection performance (as shown in Fig.3(d)).

Fig. 5
Fig. 5 Two-dimensional measures for these extracted rocks.Results show ranking of each rock for two measures.
rocks and their associated fitting ellipses.Fitting ellipse: red dashed lines, rock boundary: yellow polygons.
is an effective implicit representation for evolving the motion of curves or surfaces in two-dimensional (2D) or threedimensional (3D) space and has been successfully applied in image segmentation problems to date since it allows for automatic change of topology, such as merging and breaking.For image segmentation, active contours implemented via level set methods can evolve from an initial position to the desired features, such as the object boundaries, in the direction normal to the active contours subject to constraints in the images.Let , let R denote the set of real number, let  denote an empty set.The goal of the level set-based image segmentation is to separate the whole image domain  by C , where C denote an active contour, let  represent the image domain, level set formulation of active contours,