OPTIMIZING THE DISTRIBUTION OF TIE POINTS FOR THE BUNDLE ADJUSTMENT OF HRSC IMAGE MOSAICS

For a systematic mapping of the Martian surface, the Mars Express orbiter is equipped with a multi-line scanner: Since the beginning of 2004 the High Resolution Stereo Camera (HRSC) regularly acquires long image strips. By now more than 4,000 strips covering nearly the whole planet are available. Due to the nine channels, each with different viewing direction, and partly with different optical filters, each strip provides 3D and color information and allows the generation of digital terrain models (DTMs) and orthophotos. To map larger regions, neighboring HRSC strips can be combined to build DTM and orthophoto mosaics. The global mapping scheme Mars Chart 30 is used to define the extent of these mosaics. In order to avoid unreasonably large data volumes, each MC-30 tile is divided into two parts, combining about 90 strips each. To ensure a seamless fit of these strips, several radiometric and geometric corrections are applied in the photogrammetric process. A simultaneous bundle adjustment of all strips as a block is carried out to estimate their precise exterior orientation. Because size, position, resolution and image quality of the strips in these blocks are heterogeneous, also the quality and distribution of the tie points vary. In absence of ground control points, heights of a global terrain model are used as reference information, and for this task a regular distribution of these tie points is preferable. Besides, their total number should be limited because of computational reasons. In this paper, we present an algorithm, which optimizes the distribution of tie points under these constraints. A large number of tie points used as input is reduced without affecting the geometric stability of the block by preserving connections between strips. This stability is achieved by using a regular grid in object space and discarding, for each grid cell, points which are redundant for the block adjustment. The set of tie points, filtered by the algorithm, shows a more homogenous distribution and is considerably smaller. Used for the block adjustment, it yields results of equal quality, with significantly shorter computation time. In this work, we present experiments with MC-30 half-tile blocks, which confirm our idea for reaching a stable and faster bundle adjustment. The described method is used for the systematic processing of HRSC data.


INTRODUCTION
In the exploration of our solar system the mapping of Mars is an important task.More than 14 years ago the European Space Agency (ESA) sent its first interplanetary mission to our neighboring planet: Mars Express was launched on June 2, 2003 and inserted into orbit around Mars on December 25 that same year.One of the scientific instruments on board the orbiter is the High Resolution Stereo Camera (HRSC).This multi-line sensor with five panchromatic and four multi-spectral CCD arrays was developed by the German Aerospace Center (DLR) specifically designed for photogrammetric mapping purposes (Neukum et al., 2004).The HRSC images the Martian surface with a nominal resolution of 12.5m per pixel.The along-track multi-stereo capability of the camera delivers image strips with three-dimensional information, which cover nearly the whole planet.The different observation angles of the panchromatic images per strip allow photogrammetric object point determination in each recorded image strip (Albertz et al., 2005).Four additional channels provide color information in red, green, blue and near infrared.For a derivation of accurate digital terrain models (DTMs) (Heipke et al., 2007) and orthophotos (Gwinner et al., 2010) the orientation data (position and pointing) of the HRSC is improved via bundle adjustment (Spiegel, 2007).This non-linear least-squares optimization uses a large number of image tie point observations to adjust the parameters of the exterior orientation (EO) of the camera.The derived DTMs and orthophotos are regularly released to NASA's Planetary Data System (PDS) and ESA's Planetary Science Archive (PSA) and are used by a large number of scientists, e.g. for geological analysis.A summary of results of the HRSC experiment has been published in (Jaumann et al., 2015).With the beginning of the systematic mapping of Mars, the United States Geological Survey (USGS) divided the Martian surface into thirty tiles (Batson and Greeley, 2007).This Mars Chart 30 (MC30) tiling scheme is now also used for photogrammetric mapping with HRSC imagery.Because of gaps in the coverage and for reasons of computational efficiency in the current processing strategy the MC30 tiles (except for the polar tiles) are split into two halves.These MC30 half-tiles define the boundaries for the generation of large DTM (Gwinner et al., 2015) and orthophoto mosaics (Michael et al., 2016).Figure 1 shows the first completed multi-orbit mosaic.This halftile is located in MC-11-E (Oxia Palus, East) and covers an area of about 2.9 million km 2 (1,330km x 1,780km).The panchromatic nadir orthophoto mosaic has a dimension of 106,500 x 142,400 pixel.The corresponding color image and the DTM mosaic consist of 26,600 x 35,600 pixel.To ensure a seamless fit the images of all strips in these half-tiles are combined to form a photogrammetric block, thus allowing a simultaneous adjustment.Compared to the adjustment of individual strips, a block adjustment reduces not only local, but also regional inconsistencies, like displacements between neighboring strips.Because of the highly elliptical orbit of Mars Express, the geometric properties of the strips, e.g.size and shape, vary widely (Figure 2a).about 1330 km x 1780 km in size; resolution: 12.5m (nadir mosaic), 50m (DTM, color mosaic); from (Gwinner et al., 2015) Additionally, the illumination conditions change from strip to strip depending on time and season when images are taken.Furthermore, the image quality is occasionally degraded by haze and sandstorms, and by oscillations of the spacecraft during image acquisition.Thus, the data set of HRSC strips for each block is very heterogeneous (see also   Also the distribution of the tie points, needed for the adjustment and found by an automated matching procedure (Schmidt et al., 2008) is heterogeneous (Figure 2b).This is caused by varying a) b) Figure 2: a) HRSC image strips in MC-11-E; b) tie points in HRSC image h1183 0000; both showing a irregular, heterogeneous distribution image texture.However, because the bundle adjustment uses the DTM derived from the Mars Orbiter Laser Altimeter (MOLA) as reference, a homogenous distribution of the tie points is preferable (Spiegel, 2007).For a systematic processing of these large blocks a manual selection of strips and tie points is obviously not desirable.In contrast to other large scale bundle adjustments problems (Hartmann et al., 2016), the image correspondence problem is already solved, due to adequate approximate values for the EO parameters.But setting up matching parameters in advance to obtain a suitable tie point distribution over all areas of the block is hard because of differing image quality.Hence a post-processing step following tie point matching is seen to be more promising.
Choosing matching parameters, that produce a large number of tie points an algorithm can subsequently optimize their distribution by reducing the number while keeping the geometric block stability.This can be carried out in image space, as done e.g. in close range photogrammetry (Barazzetti et al., 2010) or, as described in the following, in object space.
In the next section of this paper an efficient way of processing large HRSC image blocks is described and two different subblock strategies together with an object space based tie point filter are introduced.Section 3 presents two setups and results of experiments to compare the two sub-block strategies and to demonstrate the benefits of a tie point filter.The findings are summarized in Section 4 together with an outlook.

Processing Overview
For the efficient adjustment of large HRSC image blocks, a twostage photogrammetric work flow has been developed (see Figure 3).First, individual strips are adjusted separately (Figure 3: left side).The panchromatic images, their nominal EO and the MOLA DTM serve as input to adjust the pointing of the EO.Initially the image data is filtered to remove noise and compression artifacts.
A detailed description of the mathematical concepts used in the bundle adjustment has been given in (Bostelmann and Heipke, 2016).Tests with the global HRSC data set have shown a distinct improvement of the EO in 96% of 3826 image strips.By adapting the distance between orientation points along the trajectory to the available image information also high frequent oscillations were removed.Because each strip is handled individually, all computations of this step can be executed in parallel.For the subsequent block adjustment, multi-strip tie points are needed.These tie points not only connect the images of one strip, but also images of strips from neighboring orbits.For matching these multi-strip tie points a sub-block strategy is used.Two options how these sub-blocks can be built are described in Section 2.2.For each sub-block the EO and the MOLA DTM are used to resample the filtered images to a common scale and projection.In these orthophotos the multi-strip tie point matching is applied.The resulting tie points of all sub-blocks are then used for the block adjustment step.Before the simultaneous bundle adjustment of the block is carried out, the tie point filter, which is introduced in Section 2.3, is employed to optimize the distribution of the tie points.A description of the other methods used in the software modules, depicted as colored, rounded boxes in Figure 3 can be found in (Bostelmann and Heipke, 2014).

Sub-block strategies for multi-strip tie point matching
Because each strip only overlaps with a limited number of other strips in a block, it is reasonable to use sub-blocks for extracting multi-strip tie points, offering among others the possibility of parallel processing.In the used concept in each sub-block one primary strip is selected.The nadir image of this strip is used as a 'master' image for image matching.In this master a regular grid of candidate points is defined.For each of these points the homologous points in the 'slave' images are searched.Thereby up to five panchromatic images for each secondary strip of the subblock, and up to four for the primary strip serve as corresponding 'slave' images.Based on these considerations a first sub-block strategy was developed (Schmidt, 2008).It was suitable for small blocks, having a regular strip arrangement, but led to limitations when processing larger blocks with an irregular arrangement of strips.For this reason two new sub-block strategies were developed and are compared here: one, pairwise, with two strips each and one using multiple strips.For a graphical explanation of these two sub-block strategies, The primary strip, containing the master image is drawn blue.In the overlapping area of each sub-block, tie points can be observed in up to ten images (five per strip).In the remaining area of the primary strip observations in up to five images are possible.Obviously, for a larger block the number of sub-blocks for both options is higher.In terms of systematic processing, the pairwise option is more practical: computer requirements for each sub-block are more predictable and the results of the sub-block matching can be re-used for other blocks in this area.However, theoretically the multi-strip option with tie points connecting all overlapping strips is preferable.The tie point filter described below works with pairwise as well, as with multi-strip sub-blocks.

AB
Both options are considered and compared in Section 3.1.

Tie point filter
The aim of filtering is to optain a more homogenous distribution, while reducing the number of tie points.This is done by using a large number of inhomogeneously distributed points as input and then discarding points which are redundant for the block adjustment.The filtering works as follows: 1.All tie points of all sub-blocks are used to calculate their 3D coordinates in a common global coordinate system together with the forward intersection error of all inherent image rays.
2. These 3D object points are clustered based on their lateral position using a regular grid.The size of each grid cell is constant and is defined by one parameter for the entire block, which is empirically evaluated in Section 3.1.
3. For each group one or more 'best points' are selected, the others are discarded.
The selected 'best points' of each grid cell need to fulfill the following two criteria: 1) together they should have image observations in the same number of strips, as the union of all points of the grid cell does, and 2) their number should be minimal.
The algorithm selects the points with the highest number of connections (if there are more than one, the point with the lowest forward intersection error is taken), until all strips are connected.In many cases one point connecting all strips (available for this set) is selected as 'best point' (see tie point 3 circled in blue in set 1).For other cells not all strips are connected by only one tie point.In these cases more than one point is selected (see point 2 and 6 in set 2).When the maximum number of connections is given for more than one tie point, the one with the lowest forward intersection error is chosen (see point 5 in set 3).
In Figure 8 a result of the filter is shown for the small block with four strips.Because for this example the pairwise strategy was used, tie points connect at most two strips (see red dots in Figure 8).Tie points marked in green connect images of one strip.As depicted in Figure 3 the tie points, selected by the filter (outlined in Figure 8) are subsequently used for the block adjustment.

Block Adjustment and Evaluation
The block adjustment uses the EO adjusted by the strip processing as input.For the camera pointing and for the height component no additional improvement is expected.Compared to the strip adjustment the stochastic model is modified in a way that only bias parameters for the lateral position of each strip are estimated.
For the evaluation of the block adjustment, the block adjusted EO is compared to the nominal and the strip adjusted EO (see Figure 3).A large number of 3D points are estimated for each case and three measures are used to evaluate the accuracy and precision of each 3D point: • The forward intersection error assesses the precision.
• The height difference to the MOLA DTM is taken as measure for the absolute accuracy.
• The forward intersection error of multi-strip tie points detects displacements between neighboring strips.
For a qualitative analysis of individual adjustment results, all three measures are used to generate error maps.This allows e.g. to detect systematic errors.For the comparison of multiple data sets, average values are more useful.

EXPERIMENTS AND RESULTS
The described method has been applied on a large number of HRSC blocks, including smaller ones for testing and improving the methods, and large MC-30 half tile blocks with up to 100 strips for the development of a new HRSC data product.
In the following, the results of two blocks are presented; firstly to validate the new tie point filter and to demonstrate its improvement in efficiency and secondly to answer two questions: • How does the choice of sub-block strategy (pairwise or multistrip) influence the results of the block adjustment?
• How does the size of the grid cells used in the filter (sec.2.3) influence the number of tie points and thus the computation time and possibly also the quality of the resulting EO?
For computational reasons elaborate parameter tests have only been carried out with a smaller block containing 19 HRSC strips (Section 3.1).The findings of these tests have subsequently been used to process the block with 98 strips from the MC-11-E halftile area (Section 3.2).

Tests on medium sized block (19 HRSC strips)
In Figure 9 footprints of the nadir images are plotted.The medium sized block with 19 strips contains 48 usable pairwise sub-blocks and 19 multi-strip sub-blocks, the latter with up to 10 strips.
16 block adjustments were carried out: eight using the pairwise sub-blocks and eight with multi-strip sub-blocks.Using the same input data only the size of the grid cells for the tie point filter was varied using the following values: 0, 4, 8, 16, 24, 32, 48, 64 (for 0, no filter was used).Because the MOLA DTM is used for evaluation, the size of the grid cells is measured in MEDGR pixel (Smith et al., 2003).One pixel refers to 463m at the equator.
Using the largest grid cell size (64 = 29,632m) the filter selects the lowest number of tie points.Figure 11 shows a result of the evaluation.The forward intersection error of multi-strip points depicts the improvement of block adjusted EO compared to strip adjusted EO.Using small grid cell sizes both sub-block strategies yield EO with the practically same precision.When using a grid cell size of 24 or larger, the intersection error increases when using the pairwise option.Using multi-strip sub-blocks the intersection error does not increase until the grid cell size is larger than 48.For reference, the red line marks the intersection error of the strip adjusted EO.Obviously an increase in the intersection error due to tie point filtering should be avoided.So a grid cell larger 48 should not be used.Further, for cell sizes of 24 and smaller no significant difference between pairwise and multi-strip sub-blocks is identifiable.The current implementation of the sub-block tie point matching is not able to handle multi-strip sub-blocks at a scale, needed for the processing of the MC-11-E block.For this reason only pairwise sub-block results are shown in this section.For this large block of 98 HRSC image strips, 368 pairwise sub-blocks have been detected.The sub-block matching delivered 910,155 tie points.The tie point filter with a grid cell size of 24 reduced this number distinctly by selecting only 5% or 44,181 points.The block adjustment was then carried out twice: once with the unfiltered and once with the filtered tie points.In Figure 12 evaluation results of both block adjustments, the strip adjusted EO and the nominal EO are compared, using error maps and mean values.
In terms of the mean intersection error of multi-strip tie points, which is taken as a quality measure for a consistent combination of strips, the block adjustment, using unfiltered tie points, achieved a reduction from 50.1m to 20.6m.Compared to the average image resolution (17.0m for the nadir images, and a factor of two to four larger for the other panchromatic images), this is a reasonable result.Although, after applying the tie point filter, only 5% of the number of tie points where used, this improvement does not suffer.However the size of the equation system to solve in the adjustment, and so the computation time decreases considerably (see

CONCLUSION AND FUTURE WORK
For the efficient systematic bundle adjustment of large HRSC image blocks a newly developed tie point filter has been added to the photogrammetric process.It optimizes the distribution of tie points by using a large number as input and discarding points, which are redundant to the adjustment.When setting up parameters of the multi-strip image matching, which influence the number of resulting tie points, a tradeoff between the computational feasibility and quality of the adjustment results had to been made.This applies for both sub-block strategies presented in Section 2.2.The tie point filter, introduced in Section 2.3 addresses this problem by selecting only tie points crucial for the adjustment and therefore allowing a faster bundle adjustment without sacrificing the geometric block stability.
Experimental results, presented in section 3.1, show that for both sub-block strategies very satisfactory adjustment results can be achieved.The theoretically less stable pairwise option yields adjustments results of equal quality as the multi-strip option, if the grid cell size is sufficiently small (compare Figure 11).The importance of optimizing the tie point distribution becomes clear, when the computation times in Table 2 are compared.As validated by the evaluation results, using the unfiltered set of tie points is not advantageous in terms of the geometric accuracy of the results, but increases the computation time from 4.5 to 222 hours.
As future work it is planned to implement the multi-strip tie point matching to handle also very large sub-blocks, as needed for the MC30 half-tiles.In this way it can be checked, whether a transfer of the results obtained in Section 3.1 with the medium sized block to a MC30 half-tile block is justified.Subsequently, the operational processing of the half-tile blocks at the equatorial region of Mars will is envisaged.

Figure 3 :
Figure 3: Process for the bundle adjustment of HRSC image data Figure 4: a) Exemplary block of four HRSC image strips (A: h1183 0000, B: h4350 0001, C: h9435 0009, D: ha469 0000); b) corresponding connectivity graphUsing this example block, ten pairwise sub-blocks can be built, as shown in Figure5.Each pair of overlapping strips, represented by an undirected edge in the connectivity graph, provides two sub-blocks by selecting each partner as primary strip once.This redundant strategy is necessary, since for the block adjustment tie points are needed in all areas of both strips, not only in the overlapping area.

Figure 5 :Figure 6 :
Figure 5: Pairwise sub-blocks for the block shown in Figure 4; blue: primary strip; orange: secondary stripUsing multi-strip sub-blocks, as shown in Figure6, each strip is selected as primary strip only once.For the same example, this results in four sub-blocks.The number of secondary strips (orange) varies, depending on the number of overlapping strips.In this way tie points observed in more than ten images are possible.

Figure 7 Figure 7 :
Figure7gives a graphical interpretation of this selection.A graph consisting of tie points and image strips is shown.Again, the image strips are labeled with capital letters (ABCD).The tie points are represented by numbers.Each set of seven tie points refers to an area marked in red in Figure4, which overlaps with two or three strips.For this reason, in each set one or two strips are not connected by tie points.

Figure 8 :
Figure 8: Outlined are 333 of 7742 tie points selected by a filter; in green: single-strip points; in red: multi-strip points; used sub-block option: pairwise; used grid cell size: 64 MEDGR pixel (approx.30km)

Fig. 10 Figure 10 :
Fig. 10 shows how the grid cell size influences the number of tie points.The unfiltered number of tie points in case of the pairwise

Figure 11 :
Figure 11: Intersection error of multi-strip tie points

Figure 12 :
Figure 12: Color-coded intersection error of multi-strip points; a) before the adjustment, mean = 50.1m;b) after strip adjustment, mean = 29.2;c) after block adjustment with unfiltered tie points, mean = 20.5m;d) after block adjustment with filtered tie points, mean = 20.6m
Statistics of 98 HRSC image strips located in MC-11-E

Table 2 :
Information about block adjustments of MC-11-ETable3summarizes the results of the block adjustments of MC-11-E compared to the nominal and the strip adjusted EO.It can again be seen that the reduction of the tie points through filtering does not have any negative effect on the results.The processing of other MC30 half-tile blocks have shown comparable results.noBA strip BA block BA (not filered) block BA (filtered)

Table 3 :
Evaluation results for MC-11-E