GEOMETRIC STITCHING METHOD FOR DOUBLE CAMERAS WITHWE A K CONVERGENCE GEOMETRY

In this paper, a new geometric stitching method is proposed which utilizes digital elevation model (DEM)-aided block adjustment to solve relative orientation parameters for dual-camera with weak convergence geometry. A rational function model (RFM) with affine transformation is chosen as the relative orientation model. To deal with the weak geometry, a reference DEM is used in this method as an additional constraint in the block adjustment, which only calculates the planimetry coordinates of tie points (TPs). After that we can use the obtained affine transform coefficients to generate virtual grid, and update rational polynomial coefficients (RPCs) to complete the geometric stitching. Our proposed method was tested on GaoFen-2(GF-2) dual-camera panchromatic (PAN) images. The test results show that the proposed method can achieve an accuracy of better than 0.5 pixel in planimetry and have a seamless visual effect. For regions with small relief, when global DEM with 1 km grid, SRTM with 90m grid and ASTER GDEM V2 with 30m grid replaced DEM with 1m grid as elevation constraint, it is almost no loss of accuracy. The test results proved the effectiveness and feasibility of the stitching method.


INTRODUCTION 1.1 General Instructions
GF-1/2 is the first one to be launched by China's high resolution earth observation project, which can provide high accuracy data and satisfy various application requirements [1].Multiple cameras are adopted by many high-resolution satellites of China to achieve a wide swath.The satellite carries two same pushbroom high-resolution cameras, which includes a PAN chargecoupled device (CCD) array and a multispectral (MS) CCD array with ground sampling distance (GSD) of 0.81m and 3.24m, respectively.The PAN consists of five linear CCD arrays placed in a noncollinear manner on the focal plane with a certain overlap cross track between adjacent CCD arrays.Figure 1 shows the geometric characteristic of GF-2 camera, the angle between cameras is 2.01 degrees and a certain overlap cross track is 380 pixels, so its convergence geometry condition is very weak.In order to improve the efficiency of the highresolution images application, it is necessary to provide a high quality image stitching product with uniform geometric precision.The seamless mosaic images without distortion are necessary for subsequent photogrammetry production, such as sub pixel stitching accuracy is the premise to ensure the accuracy of DEM production.The problem of how to make the high precision geometric stitching of the multiple cameras images becomes an urgent problem needed to be solved.[3][4].However, the potential error sources of the method are analysed deeply.The stitching error caused by elevation error cannot be ignored, especially for the elevation more than 300 meters,it will cause 1 pixel stitching error in push-broom direction [5].
In view of the above problems, and taking into account the small converge angle between the camera images, it may cause erroneous results or iteration failures during the block adjustment.We proposed a new geometric stitching method, which utilizes DEM-aided block adjustment to solve relative orientation parameters of each camera image for dual-camera with weak geometry.A RFM with affine transformation model is chosen as the relative orientation model.To deal with the non-convergence problem caused by weak geometry, a reference DEM is used in this method as an elevation constraint in the block adjustment, which only calculates the planimetry coordinates of TPs, then using the obtained affine transform coefficients to generate virtual grid, then updating the new RPCs to complete the geometric stitching.

RFM with Affine Transformation Model
Satellite photogrammetry has the characteristics of long focal length, narrow FOV and nearly parallel projection.Highresolution satellite image processing method are mainly based on the theoretically rigorous imaging geometry model, but there will be a strong correlation between the orientation parameters.Thus, its superiority has often been solved by many parameters, and the unstable numerical solution [6].RFM is well known and widely used in the distribution and processing of satellite imagery, which has been proven to be very simple, fast, and stable.So, RFM is selected as the geometric model.RFM uses the ratio of two cubic polynomials to describe the relationship between the image space and the object space as shown in (1).To maintain numerical precision, the object and image space coordinates will be normalized to (−1, +1) [6][7].
( , , ) ( ) To compensate for the systematic error of the RPCs, we use an affine transformation model to correct and refine RPCs in the image space [8].The coefficients for the affine transformation can be calculated from the GCPs.The equation for the affine transformation is x e e x e y y f f x f y where V = residual matrix for the RFM equations A = the design matrix which consists of partial derivatives of x, y to a0, a1, a2, b0, b1, b2,respectively B = the design matrix which consists of partial derivatives of x, y to X, Y, Z, respectively s = the correction vector of affine parameters t = the correction vector of the unknown GCPs l = the difference of the calculated and observed image point coordinates

DEM-Aided Intersection for Ground Points
To deal with the weak convergence geometry and small base-toheight ratio situation, a reference DEM is used as an additional constraint for the ground points.First, an initial elevation 0 Z is given, and the corresponding planimetry coordinates

Update RPCs
We can solve the affine transformation parameters through iteration, and then updating RPCs.Specific steps are as follows: 1. Using the original RPCs and affine transformation parameters to generate the new virtual control points grid without system error, grid size is 10*10*5; 2. Using the virtual grid, update the new RPCs according to the L curve method [11].
3. Using the new parameter for the precision statistics in the object space and image space.

Data Set
The test data were GF-2 level 1A standard products, including images and the corresponding RPCs.The geometric characteristic of GF-2 dual-cameras shows that the angle between cameras is 2.01 degrees, so the convergence geometric condition is very weak.

Test scheme
In order to compare and verify the accuracy of the stitching algorithm in different regions and different terrain condition, this paper mainly uses the following test scheme.

The accuracy of stitching with and without GCPs:
The images of Henan area are used to stitch according to experiment 1.The results of the test are shown in Table 1.As shown in Table 2, the RMSE of TPs without GCPs is about 0.48 pixels and the RMSE of TPs with GCPs is about 0.65 pixels.The accuracy of stitching with GCPs is slightly lower than without GCPs.The artificial measurement accuracy of GCPs is difficult to achieve sub-pixel, therefore result in a slight inconsistency in precision.In the course of adjustment, the poor accuracy GCPs will affect the network of the whole area.We can solve the affine transformation parameters through iteration, and update RPCs.Evaluating the stitching accuracy of images with new RPCs by overlapped display.The images are rescaled necessarily for a better visual effect.The stitching effect of Henan area is shown in Fig. 3.It can be seen that the accuracy of the overlap area has a seamless visual effect and there is no obvious misalignment phenomenon in the areas(roads, houses, bridges, mountains, and so on).The method proposed in this paper can ensure higher geometric consistency in the overlapped area, and skip the step of stitching line selection The images of two areas (Beijing and Henan) and different accuracy DEMs are used to stitch according to experiment 2. As shown in Table 3, it is almost no loss of accuracy for Henan test area, when global DEM with 1 km grid, SRTM with 90m grid and ASTER GDEM V2 with 30m grid replaced DEM with 1m grid as elevation control.Similarly, it can achieve consistent geometric accuracy for Beijing test areas.This is because the geometric characteristics of GF-2 dual-camera, which two cameras approximate vertical observation, projection error is very small for regions with small relief, so different DEMs have the same elevation control effect.2 shows the visual effect of stitching with different DEMs-aided.In order to better show the effect visual of stitching, the image display grayscale.It can be seen that the accuracy of the overlap area has a seamless visual effect and there is no obvious misalignment phenomenon in the areas (roads, houses, bridges, mountains).

CONCLUSION
In this paper, a new geometric stitching method is proposed which utilizes DEM-aided block adjustment to solve relative orientation parameters of each camera image for dual-camera with weak convergence geometric.The panchromatic images of GF-2 dual-camera were tested and compared the stitching accuracy with different DEMs.After the full analysis of the experiment results, we conclude the following.
1.The proposed method can achieve an planimetry accuracy of better than 0.5 pixel of the TPs and has a seamless visual effect.The test results proved the effectiveness and feasibility of the stitching method.
2. The non-convergence caused by weak convergence geometric is solved by the reference DEM in the block adjustment.
3. For regions with small relief, it is almost no loss of accuracy for test areas, when global DEM with 1 km grid, SRTM with 90m grid and ASTER GDEM V2 with 30m grid replaced DEM with 1m grid as elevation control.The method proposed in this paper can ensure higher geometric consistency in the overlap area, and omit optimal stitching line selection step.

Figure 1 .
Figure 1. the geometric characteristics of GF-2 dualcamera High-resolution satellites rarely use multiple cameras to achieve a wide swath and relevant technical documents and literature are found few.Several researchers focused on image stitching with weak geometry.Jcobsen et al. proposed a simple image shift method to stitch for the three CCD line-sensors of the IRS-1C satellite panchromatic camera, and the accuracy of the model is from 0.2 to 0.6 pixels in image space [2].Zhang and Tang et al. used an inner field of view (FOV) stitching method based on virtual CCD line, ZY-3 and ALOS/PRISM images are used to verify the algorithm [3-4].However, the potential error sources of the method are analysed deeply.The stitching error caused by elevation error cannot be ignored, especially for the elevation more than 300 meters,it will cause 1 pixel stitching error in push-broom direction[5].In view of the above problems, and taking into account the small converge angle between the camera images, it may cause erroneous results or iteration failures during the block x y = image point coordinates by RPC ( , ) x y = the observations of the image point coordinates 2 0  e f = affine coefficients The error equation can be established as follows: X Y are calculated by RPCs and affine transformation parameters.We interpolate a new elevation 1 X Y , Through iteration and interpolation, when the incremental value of the new elevation 1 than a certain threshold, we attain an accurate result and overcome the problem of elevation error.The full procedure is demonstrated in Fig. 2[9-10].

Fig. 2
Fig.2 DEM-Aided Intersection for Ground Points Images of two areas (Beijing and Henan Province in China) and the corresponding DEMs are introduced.The range of Beijing area is 116.02 °E ~116.48 °E, 39.71 °N ~40.02 °N.The terrain of the test site has about 500m relief.The range of Henan area is 112.91°E~113.37°E ， 34.21 °N~34.53°N.The terrain of the site has about 1200m relief.This test area set up 21 ground control points (GCPs).The control data is extracted by artificial measurement from the reference digital orthophoto map (DOM) and DEM.The accuracy of the DEM is about 1m in height, and the planimetry accuracy of the DOM is about 1 m.

Experiment 1 :
The images of Henan area and the corresponding 1m resolution DEM are used with and without GCPs.The purpose is to verify the planimetry accuracy of method proposed in this paper.Comparing and analysing the accuracy of with and without GCPs by calculating the TPs residual error statistically in image space, and evaluating the stitching effect by overlap display.Experiment 2: The images of two areas (Beijing and Henan) and different accuracy DEMs (include global DEM 1 km grid, SRTM with 90m grid and ASTER GDEM V2 with 30m grid) are used without GCPs.The purpose is to verify the accuracy of DEM influence on stitching accuracy.

Figure 3 .
Stitching effect of Henan test area.(a) and (b) across track stitching effect, and (c) and (d) along track stitching effect 3.3.2The stitching accuracy with different DEMs test:

Figure 4 .
Stitching effect of test areas.(a) across track stitching effect with 30m DEM, (b) across track stitching effect with 90m DEM, (c) across track stitching effect with 1km DEM, (d) along track stitching effect with 30m DEM, (e) along track stitching effect with 90m DEM, (f) along track stitching effect with 1km DEM

Table 1 .
Accuracy of stitching with and without GCPs

Table 1 .
Accuracy of stitching with and without GCPsAs shown in Table1, the planimetry accuracy of stitching is 69.59m without GCPs, this result agrees with the previously reported accuracy.While the planimetry accuracy of stitching is 1.63m with 9 GCPs and the system error is eliminated effectively.To verify the geometric consistency between images, the TPs residual in image space are statistics, as shown in Table2.

Table 3 .
Results of stitching with different DEMsFigure