VALIDATION OF THE STEREO OBSERVATION STRATEGY OF SIMBIO-SYS USING A VIRTUAL SIMULATOR

The Spectrometer and Imagers for MPO BepiColombo Integrated Observatory SYStem (SIMBIO-SYS) is a suite of three independent optical heads that will provide images and spectroscopic observations of the Mercury surface. With the approaching of BepiColombo to Mercury, the definition of the observation strategies of each instrument is becoming mandatory also for testing the operation procedures in terms of feasibility and stereo performance. The use of synthetic images and a customized simulator have been considered a powerful way to accomplish this test. This simulation system allows to investigate all the possible stereo configurations (different stereo angles and different image combinations) with the opportunity to evaluate the operational feasibility thanks to the evaluation of the final stereo products. Working with a simulated dataset allows to control most of the geometrical aspects (both the projection model definition and the observation geometry) suggesting with the analysis of the stereo products the better configuration to be applied and to be considered in the definition of the observation strategy.


INTRODUCTION
BepiColombo is the first ambitious, multi-spacecraft mission to Mercury planned by European Space Agency in co-operation with Japanese Space Agency JAXA. It has been launched in October 2018 from Kourou, French Guiana, starting a 7-year journey, which will bring the two modules to the innermost planet of the Solar System. In order to analyse the geological and mineralogical characteristics of the Mercury surface, the MPO (Mercury Planetary Orbiter) module will have on board, among other instruments, SIMBIO-SYS (Spectrometers and Imagers for MPO BepiColombo -Integrated Observatory SYStem) (Cremonese et al 2020), an integrated package which has three different optical channels. The package includes the stereo capability of two channels. The Stereo Camera (STC) has been designed with the objective of providing the 3D global mapping of the Hermean surface but also the High resolution channel (HRIC) will be also able to acquire stereo pairs for high resolution DTMs (Digital Terrain Models). The stereo products provided by the two channels will offer to the scientific community high valuable data of paramount importance for the surface morphology analysis with a better definition of the stratigraphy of the main geological units, structural features linked to global and local tectonics, impact crater populations and volcanic edifices. Among the stereo validation activities executed with the flight model in laboratory (Re et al , 2015 and2017) has demonstrated the actual capability of STC to provide three dimensional products from the stereo pairs, the stereo system performances can also be demonstrated in a virtual environment as proposed in this paper. The present work describes the customized simulator that has been settled to evaluate the possible stereo observing strategies of the SIMBIO-SYS package providing some guide lines and constraints for the future scientific observation campaigns.
The SIMBIO-SYS package is described in the next paragraph with particular interest in the stereo capabilities of the system. The virtual simulator is shown in the paragraph 3 with the definition of the acquisition geometry and the definition of the main objects involved in the simulation with SurRender. The discussion on the results in terms of vertical accuracy of the stereo products conclude the paper.

SIMBIO-SYS
SIMBIO-SYS is a complex instrument suite on board MPO and it is composed by a High Resolution Imaging Camera (HRIC), a STereo Channel (STC) and a Visible and Infrared Hyperspectral Imager (VIHI). The baseline approach used in the SIMBIO-SYS design is oriented to the best sharing of tasks between the different optical channels for imaging and spectroscopic observations. To achieve the best coordination, the proposed instrument concept is based on a highly integrated suite. This approach is effective in obtaining maximum scientific return, with respect to the goals attributed to the imaging and spectroscopic Hermean exploration, while minimising the overall resource allocation. All the data that will be acquired will allow to cover a very wide range of scientific objectives, from the surface processes and cartography up to the internal structure and the surface-exosphere interaction. The global 3D and spectral global mapping will allow to study the morphology and the composition of any surface feature.

Stereo peculiarities
The STC channel will deliver a global DTM with a grid resolution better than 120m/pixel, while HRIC will provide updates of the reference system and prime meridian by selecting and observing specific topographic targets during the mission. Both channels will concur in creating a global grid for the BELA range measurements between the spacecraft and the Hermean surface (Cremonese et al, 2020). The stereo observations are fundamental to the comprehension of Mercury interior and surface. Surface morphology will be analysed both with the bi-dimensional images at different scales and resolutions and, in addition, the 3D features will take advantage from using the STC and HRIC stereo-datasets. The STC global shape observations and the HRIC targeted high resolution images will represent an important step forward in the geomorphological characterization of the Hermean surface, also providing topographic and high resolution information on the southern hemisphere, which has been less covered by MESSENGER.

STC and HRIC channels
STC represents one of the first push frame stereo camera on board a planetary mission; the actual 2D images of the planet surface are acquired, then buffered and read while the spacecraft moves. Based on a new concept, STC integrates the compactness of a single detector telescope with the photogrammetric capabilities of a two directions camera. Two separate incoming optical paths, oriented at + and -20 degrees with respect to nadir and along the orbit path, allow the instrument to acquire images of the same surface region from two different point of views with a focal length of 95 mm. The camera has the capability of imaging in five different spectral bands: one panchromatic and four broad-bands filters, in the range between 410 and 930 nm. The stereo pair is acquired through two panchromatic filters, one for each sub-channel with spatial resolution of 60 m/pixel at 480 km from the surface. The detector is a hybrid SiPin CMOS bi-dimensional array and the area covered by the image of each panchromatic filter is 896X384 pixels, cross track and along track respectively.
The High Resolution Imaging Channel has been designed in order to satisfy the primary scientific requirement of performing imaging of selected areas of the planet (potentially over 20 % of the surface) with ground pixel size of 6 m / pixel at a distance of 480 km from the planet surface. To achieve this goal within allocated resources a field of view of 1.47° cross-track has been chosen optimising the optical design. The optical design is based on a catadioptric solution (i.e. lenses and mirrors) with a focal length of 800 mm, in order to achieve the required on ground resolution with a 10 μm pixel-size. The detector is the same for STC with a region covered by the panchromatic filter of 2048X640 pixels and other three colour filters centred at 550, 750 and 880 nm.

Observation Strategy
The orbital period of Mercury (one Hermean year) around the Sun is 88.0 Earth days, while the rotation period around its axis is 58.6 Earth days according to the 3:2 spin-orbit resonance. Every two Hermean years (176 days) the same side of the planet faces the Sun at perihelion, and the planet rotates 3 times around its axis. STC will work in the range of Mercury true anomalies between 138° and 222° (i.e. around aphelion when the MPO will have the periherm above the sunlight hemisphere). During the Stereo Global Mapping, that will be performed in the first 6 months of the nominal mission, the entire surface of the planet will be covered by both panchromatic filters. At the first periherm (480 km) the baseline of the stereo acquisition block will be 455.7 km potentially providing DTMs with vertical accuracy of 80 m (assuming a conservative 1 pixel accuracy in the image matching). Since STC has a FoV of 5.38, an overlapping between adjacent orbits will be present and will facilitate the mosaicking procedures. The Color Mapping will be performed in the second six months of the nominal mission and will provide targetoriented acquisitions of multiband images with the four broadband filters. HRIC will operate during the BepiColombo nominal mission to perform targeted high-resolution observations of specific areas of interest selected according to the scientific objectives. Almost the 20% of Mercury surface will be covered and in principle, HRIC will operate in a wide range of true anomalies of Mercury; several ranges of solar incidence angles and latitudes will be taken into account according to the specific objective of the observations. HRIC will routinely operate covering a specific target with the panchromatic filter in push frame mode but also the colour mode acquisition of the same region could be executed with the other three colour filters. HRIC can also acquire stereo pairs thanks to the possibility to observe the same surface area with different stereo view angles. Being rigidly integrated on the S/C, off-nadir pointing is needed to acquire the target by a roll or a pitch of the S/C during same orbit or the following. These special pointings need to be carefully planned to minimise the impact on the other nadir pointing instruments and global mapping of the other two channels.

THE VIRTUAL SIMULATOR
Approaching Mercury, the definition of the observation strategies of each instrument is becoming mandatory also for testing the operation procedures in terms of feasibility and stereo performance. The use of synthetic images and a customized simulator have been considered a powerful way to accomplish this test. The simulation tool that we have adopted is based on the image rendering software SurRender developed by Airbus (Brochard, R. et al., 2018). In SurRender environment, the virtual cameras can be placed, the satellite constraints and the illumination conditions can be defined and a specific 3D-model that emulates the target (Mercury) of a possible mission is introduced. The simulation results are used to perform a validation of the stereo performance of SIMBIO-SYS (both for the STC and HRIC channels) on flight and to define the stereo acquisition possibilities that the instrument can achieve exploiting both the STC stereo data and the high resolution images. The simulation work will operate at different levels: • simulation of the imaging systems, reproduction of the cameras characteristics (focal length, sensor dimensions, spectral information definition) • simulation of flight trajectories for the MPO spacecraft through the implementation of predicted SPICE kernels (Acton, 1996), allowing to take into account the impact of the attitude on the instrument design and observing strategy; • introduction into the scene of 3D-model reproducing Mercury considering the spectral specifications of surface angular reflectance distribution (BRDF) defining in the meantime several illumination conditions for multiple phase of the mission.

SurRender software
Differently from the majority of well-established software know to the public, the Airbus image rendering SurRender software provides images with high photometric accuracy. The software include physical models such as the Bidirectional Reflectance Distribution Function (BRDF) that allow good representation of the rendered scene and the image is rendered in physical units: each pixel contains an irradiance value expressed in W/m². The combination of the high resolution of the 3D model (gridded Digital Terrain Model or 3D mesh) with the accuracy of the physical model allows to generate virtual images with extremely high representativeness almost undistinguishable from real pictures (Brochard, R. et al., 2018).

Inputs
The imported meshes want to reproduce the surface of Mercury. The unavailability of high resolution DEMs of the Hermean surface bring to use some Lunar data as analogues for our purposes. The 3D models have been imported in SurRender in the triangulated mesh format (obj). For the simulations, a "generic sensor" was used. The model includes pupil and lens size, quantum efficiency, lens transmittance, gain bandwidth and integration time, readout noise and (single) PSF is also accounted for. The projection model is that of a pinhole camera, resulting in images free from radial and higher order distortions. In addition, the diffuse Hapke BRDF (Hapke 1981) was added to allow more accurate shading of Lunar regolith, where the "opposition effect" introduces a strong backscattering component.

Simulation of Acquisition Geometry
In SurRender external scripts in Python 3 describe the scene and its objects: camera, light sources, targets, their positions, attitudes, shapes and surface properties. The observation geometry has been re-created taking advantage of the complete set of BepiColombo SPICE data files (kernel files), which can be accessed using SPICE software. The SPICE data contain geometric and other ancillary information needed to recover the full value of science instrument data. In particular SPICE kernels provide spacecraft and planetary ephemeris, spacecraft and instrument orientation, instrument mounting alignments, and data needed for relevant time conversions. Data in the SPICE kernel files must be accessed using the software called the SPICE Toolkit produced and distributed by the Navigation and Ancillary Information Facility (NAIF) Node of the Planetary Data System. This SPICE archive includes data from the BepiColombo launch, up to through the end time of the latest spacecraft orientation file supplied in the archive. Until the end of the spacecraft lifespan this archive will be accumulating with new data added according to the project archiving plan. The SPICE toolkit has been implemented in a python scripts in order to: • define the geometry acquisition (position and attitude of the main element frames and position of the Sun for the definition of the illumination conditions) • derive the camera matrix and the projection matrix useful for the initialisation of the photogrammetric process for the threedimensional reconstruction.
The geometry starts with the access to the BepiColombo metakernel and the definition of the camera frame that we want to reproduce. All the camera parameters essential for the definition of the camera model has been accessed through the SPICE functionalities as well as the exterior orientation parameters (the translations and attitude angles of the perspective center). Using the SPICE toolkit functionality we are able to reproduce with high accuracy: the attitude and the position of the camera frame with respect to the surface and the illumination conditions. This scenario, reproducing perfectly the acquisition geometry, the influence of the pointing accuracy and the influence of the illumination conditions on the final stereo results (also with respect to the surface characteristics) allows to define a very good observation strategy.

Data sets
For the HRIC simulation we used lunar DTMs based on data from the NAC system for the LROC (Lunar Reconnaissance Orbiter Camera) (Robinson, 2009) on the NASA's Lunar Reconnaissance Orbiter (LRO) instrument (Vondrak et al. 2010) downloaded from (http://wms.lroc.asu.edu/lroc/rdr_product_select). The GSD (Ground Sample Distance) is around 2 m. We assume that the BRDF of the Moon is best described by an Hapke model. The NAC_DTM_A17SIVB_E042S3476 DTM model has a size of 2541x14340 pixels and covers an area of 5082x28680 m.
Synthetic images were then generated from the 3D model at different locations along the BepiColombo orbits. The same DTM has been located at different positions (28°, 60°,80° Latitude) accurately reproducing the acquisition of the surface from the MPO orbit defined by the SPICE kernels. a) b) c) d) Figure 4. Crops of the simulated images virtually acquired at a) Latitude 28° and incidence angle 33°, b) Latitude 23° and incidence angle 72°, c) Latitude 60° and incidence angle 65°, d) Latitude 80° and incidence angle 82° For the simulation of the STC acquisition still a lunar DTM was used but produced, in this case, starting with the Japanese lunar explorer Kaguya (SELENE) (Kato, 2010) images. The DTM used is a crop from the DTM_MAP_02_N00E030S03E033SC (https://darts.isas.jaxa.jp/planet/pdap/selene/) of 1314x1452 pixel size, GSD is around 14,8 m/px covering an area of 19447x21489 m^2. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2020, 2020 XXIV ISPRS Congress (2020 edition) Both the HRIC camera and the STC stereo acquisitions have been reproduced in order to evaluate the stereo performance of the SIMBIO-SYS suite. In the HRIC test, an extensive generation of simulated stereo pair images taken with different geometries (two emission angles and four incidence angles) was performed using the same DTMs. A specific code has been developed to position the DTMs on a particular location as on the surface of Mercury. Given the Latitude and Longitude, the three dimensional model is located in that region in order to reproduce any geometrical feature imposed by the mission in terms of observation and illumination.
The solar incidence angle depends on two variables:  From the graph suggests that, at low Latitudes, the differences in the solar incidence angles are more evident; the procedure of pairing imaging is more affected by the change of the solar inclination at low Latitudes than at high latitudes and the polar regions are characterized by high incidence angles. The incidence angles can introduce issues when the matching images are taken at different periods of time. Indeed, the different illumination conditions of the scene, that affects the entity of the shadows, could compromise the efficiency of stereo pairing or mosaicking. For along track acquisitions this problem becomes irrelevant, as two consecutive images are taken in a short difference of time, but it is more significant for across track acquisitions, where consecutive images are taken with long difference of time (could be different months). For this reason, higher attention must be put into carefully planning across track acquisitions.
Solar incidence can affect different areas in different ways in dependence to the morphology of the target. Regions with strong altitude differences produce shadows with shapes and dimensions dependent to the actual position of the Sun, while this effect is less significant for flat regions. Thus, for a complete evaluation of the phenomenon, DTMs with different morphologies have been selected.

DISCUSSION
Since one of the main scientific aim of the BepiColombo mission is to provide a global topographic mapping of the entire Herman surface with a vertical accuracy of around 80 m and the 20% of the whole coverage with high resolution imaging, the careful design and stereo imaging planning become mandatory to allow accurate stereo products. The mission foresees to achieve local and regional highresolution DTMs and image mosaics to be complementary with contextual and global shape of the planet derived by the Stereo Camera.
Since the final mission timeline and orbit predictions are available, we still need to study more carefully illumination conditions and the possible off-nadir pointing angles that represent a good tradeoff between the estimated vertical accuracy and the mission constraints. The polar region, furthermore, are challenging due to the possibility to exploit multiple imaging with several solar azimuths but significant shadowed regions which can affect the image matching process. In order to extract useful indications for the observation stereo planning, the analysis of the DTMs (obtained changing the geometric and illumination conditions and the morphology) in terms of vertical accuracy has been considered.

DTM Generation
The DTMs has been generated through the 3DPD software developed at the Astronomical Observatory of Padova (Simioni, 2017). The pipeline contains some sequential steps: the creation of the map-projected mosaics from the framelets, the definition of an initial disparity map, the disparity refinement at the subpixel level and the triangulation phase arriving to the DTM production. The software implements a sub-pixel refinement by area based matching (Least-Squares Matching) is execute (Gruen, 1985).

Evaluation of Vertical Accuracy and Expected Vertical Precision
DTM and its derived attributes (slope, curvature, topographic index, topographic sections etc.) are important parameters for information extraction or assessment of any process using terrain analysis (Wolock and Price, 1994). The validation was performed collecting the statistics of the discrepancies between the DTM, photogrammetrically obtained and the DTM that was used to simulate the ground truth. The statistics have been collected in terms of root mean square error (RMSE) and mean error (ME). RMSE exhibits on average how far observed values differ from the assumed true value. The ME tells us whether set of measurements consistently underestimate (negative ME) or overestimate (positive ME) the true value. The RMSE is a single quantity characterizing error surface, and mean error reflects the bias of the error surface (Mukherjee et al., 2011). The equations as follows: At the same time, the expected vertical precision (EP) can be evaluated from the simplified theoretic formula (: Where Dp is the RMS stereo matching error in pixel units (0.2 pixel often used as a rule of thumb (Cook et al., 1996)), H is the height of the Center of perspective, B is the baseline of the stereo block, c in the focal lengths.

Effect of illumination on DTM accuracy and on Matching process
A better knowledge of the effect of the illumination on the final stereo products has been extracted from the statistics coming from the comparisons between the DTMs photogrammetrically obtained from the synthetic datasets and the reference DTM (used for the image generation) in terms of RMSE and ME; at the same time, the statistics related to the matching performance has been considered.
The mean values (ME) of the disparities along the x axis (Di), the mean values of the disparities along the y axis (Dj), the Standard Deviation (STD) of both the disparities, the mean values of the Normalised Cross Correlation (Lewis, 1995) coefficients (index of the performance of the matching process) have been considered in the evaluation. In the theoretical formula, however, the matching error has been fixed at 0.2 pixel. The RMSE results suggest, instead, that the lower is the emission angle, the higher is the vertical accuracy. The matching performance parameters help in the clarification of this behaviour. Indeed, especially when the incidence angle is low as Table 1 a. and b., an increase of the emission angle results in a certain difference in perspective that make difficult the matching process demonstrated also by higher NCC of the related to results of 5° angle with respect the results obtained with the 10° emission angle.
It is worth to clarify that when the images are acquired at low Latitude values (Figure 4 a,b), it means that the overall illumination of the image creates low contrasts. Even though the computed incidence angle is high as in Figure 4 b; in this case, the direction that define the incidence angle is longitudinal and the difference between the two acquisitions (Fig 4a and b) has been defined by two different True Anomaly. The first two case (position 1 and 2) are characterised by difficulties in the matching process with respect the other last two positions where the Latitude is increased and the matching algorithm is facilitated in the identification of the conjugated points. Shadows begin to affect stereo reconstruction significantly at latitudes beyond ~60°, in those areas many scientific targets has been already selected furthermore, the regions around 90° has been suggested to be deposits of water ice trapped in cold, permanently shadowed regions. Though the smallest stereo reconstruction errors are obtained at a low Sun (ϑ > 60_) due to the high image contrast, here the shadowing starts to play an important role as discussed above. (I. Polyansky et al 2018). -0.93

ME (NCC) 0.975
Also from the analysis of the STC case, in terms of results on the image plane (matching process), the NCC coefficient is a bit lower than the results obtained at the same Latitude and the same incidence angle with the HRIC images acquired with lower emission angle. At the polar regions, in fact, the convergence angle for STC increases due to the orbit ellipticity and the accuracy varies along a meridian as a function of latitude (Massironi, 2008). At the polar regions becomes interesting the integration between the two channels of SIMBIO-SYS because where the STC accuracy decreases, HRIC can exploits the multiple acquisitions on the poles where also multiple convergence angles can be considered.

CONCLUSION
The simulation system arranged within the SurRender platform allows to investigate all the possible stereo configurations with the opportunity to evaluate the operational feasibility thanks to the evaluation of the final stereo products. Working with a simulated dataset allows to control most of the geometrical aspects (both the projection model definition and the observation geometry) suggesting with the analysis of the stereo products the better configuration to be applied and to be considered in the definition of the observation strategy. The possibility to exploit the SPICE kernels has allowed to accurately reproduce most of the geometrical aspect of the orbit acquisition. The illumination conditions can be investigated with this virtual configuration together with the possibility to simulate different epochs (True Anomalies and location with respect the surface of the planet) and different stereo angles. All the parameters considered help in the analysis of the best configurations for the definition of the operation strategies. What can be summarised, from this preliminary work, is that when a possible off-pointing has to be planned, in order to obtain the best stereo products, both geometric aspects (as the convergence angle, emission angle and baseline) and illumination conditions have to be investigated.
A stereo block, geometrically robust, can reduce the final vertical accuracy if the matching process is compromised by the bad illumination conditions that cause low contrasted features.