MULTIMODE SATELLITE IMAGE HYBRID BLOCK-ADJUSTMENT AND ITS APPLICATION IN LARGE AREA ORTHOPHOTO IMAGE PROCESSING

Different satellite images have different positioning accuracy. For example, stereo satellite images have higher positioning accuracy than resource survey satellite images. In addition, for a large number of non stereo satellite images, due to the inability to build a strong triangulation model, it is impossible to carry out block adjustment alone to improve the image positioning accuracy. High precision and high resolution Orthophoto Images are the basis of resource investigation and monitoring and basic geographic information updating. For example, China's third national land survey, national geographic situation monitoring and other national projects require that the survey base map must reach the accuracy of 1:10000 scale, that is, the mean square plane error of points is less than 5m. For most satellite images, a certain number of ground control points need to be deployed to achieve this accuracy. Due to the difficulty of obtaining high-precision ground control points and DEM data in difficult areas, high-precision mapping has always been an unsolved problem, such as Western China. In addition, due to the limited coverage of a single satellite image, to realize the complete coverage of an image in a large area requires the joint application of multiple satellite images. In this paper, the high-precision collaborative geometric processing model and technical method of high-resolution multi-source remote sensing satellite images are proposed. The high-precision collaborative geometric processing of more than ten kinds of high-resolution domestic satellite images is completed by integrating multi-source observation data. An automatic construction method of large-scale block adjustment model of remote sensing images from domestic satellites based on multivariate generalized control network is proposed, including key technologies such as automatic optimization of optimal tie points under different modes, automatic matching of multi-node parallelization tie points, multilevel gross error elimination and so on, which realizes the automatic and stable construction of aerial triangulation model. The test shows that the positioning accuracy of satellite images is better than the accuracy requirements of 1:10000 scale without ground control points, which solves the problem of geometric positioning of 1:10000 scale accuracy in areas where it is difficult to obtain ground control points in the field of Western China.


INTRODUCTION
Different satellite images have different positioning accuracy. For example, stereo satellite images have higher positioning accuracy than resource survey satellite images , Song Y et al., 2013, Zhong Bin et al.,2016. The Orthophoto Image with high geometric positioning accuracy and high resolution is the basis of resource investigation and monitoring. For example, China's third national land survey project requires that the orthoimage must reach the accuracy of 1:10000 scale, that is, the mean square error of the image is less than 5m. For most satellite images, a certain number of ground control points need to be deployed to achieve this accuracy (Li J et al., 2006, Qin X W et al.,2007. In addition, for non-stereo satellite images, because the elevation constraint cannot be realized in aerial triangulation, the image positioning accuracy cannot be improved by aerial triangulation adjustment.
Due to the lack of ground control points in some areas, highprecision mapping using satellite images has always been an unsolved problem, such as the mountainous areas in west of China. Because the positioning errors of different satellite images vary greatly, it will be more difficult to achieve the above accuracy for a variety of non stereo images at the same time.
In this paper, the process flow of high-precision hybrid aerial triangulation of high-resolution multi-source remote sensing satellite images is proposed, and the high-precision hybrid blockadjustment processing of more than ten kinds of satellite images is completed by integrating multi-source observation data. An * Corresponding author automatic construction method of large-area triangulation network of heterogeneous satellite remote sensing images based on generalized control network is proposed. It includes key technologies such as automatic optimization of tie points under different modes, parallel automatic matching of tie points, multilevel gross error elimination and so on. The positioning accuracy is better than the accuracy requirement of 1:10000 scale, which solves the problem that it is difficult to measure the ground control points in mountainous areas in West of China.

Automatic extraction and optimization of tie points
Firstly, we need to automatically extract the tie points between overlapping images through image matching, and build the topological relationship model between images to be adjusted based on these tie points to realize the automatic construction of triangulation network. For the various sensor images, stereo images and non stereo images, and the irregular large aerial belt structure, the traditional automatic modeling method based on the regular zone structure is no longer applicable. How to efficiently obtain the tie points between a large number of images, automatically select the tie points with high reliability, high overlap and optimal point distribution from these tie points, and then automatically build the connection topology relationship model with sparse structure based on these tie points is the key problem to be solved in this method (Wang M et al.,2016, Yang Bo et al.,2017.
In this paper, image matching based on SIFT operator is used to extract image tie points. Based on this result, least square image matching is used to optimize the accuracy, which not only solves the reliability problem of tie point extraction caused by obvious feature differences among multi-source images, but also realizes the measurement accuracy of tie points required by aerial triangulation adjustment and meets the requirements of sub-pixel accuracy.
Before adjustment, the tie points need to be optimized to avoid the problems of low calculation efficiency and slow convergence caused by repeated observation results. The automatic optimization of tie points is mainly divided into two steps. Firstly, the redundant tie points are eliminated, and then the gross error is detected and eliminated(Yang Bo et al., 2017). The specific process is shown in the figure below.

Hybrid aerial triangulation adjustment
Because most satellite images are attached with rational polynomial parameters (RPC), the error equation in this paper is also rational function model (Tao C V et al.,2001, Grodecki J et al.,2003. Under the condition of no control points, due to the lack of constraints of control points, the unknown number degree of freedom of the adjustment model is too high. Directly solving the parameters to be adjusted as free unknowns will lead to the ill condition of the normal equation matrix, which makes the adjustment accuracy unstable and the error is easy to accumulate excessively, resulting in the distortion and deformation of the block adjustment model. For this problem, the traditional method improves the state of the adjustment model by processing the parameters to be adjusted into weighted observations according to the prior information and introducing them into the block adjustment model. However, this method is limited in practical application because it needs to construct error equations and determine the weight for many kinds of parameters with different physical meanings and correlations. To solve this problem, this paper uses the initial RPC model of each scene image to generate virtual control points, and introduces the virtual control points instead of the parameters to be adjusted into the adjustment model as weighted observations to improve the state of the adjustment model (Wang M et al.,2016, Yang Bo et al.,2017.
The block adjustment model is an RPC model with additional parameters. The unknowns to be solved include 6 additional error compensation parameters of each image and the three-dimensional coordinates of all tie points. The adjustment formula and steps are as follows: Add the affine transformation model to the RPC model image side of each scene to be adjusted, and the adjustment model formula is: For the virtual control point, because its corresponding object point coordinates are accurately known, the unknown parameters in the constructed error equation only include the RPC model image side additional parameters of the image where the image point is located. Obviously, for the RPC model image side additional parameters, at this time, the above formula is a linear equation without linearization.
The observation error equations are constructed for all tie points and control points respectively. According to the principle of least squares adjustment, the observation error equation is normalized, and the normal equation can be obtained, as shown in the formula: In large-scale block adjustment, due to the large number of images and tie points involved in the adjustment and the high order of the coefficient matrix of the normal equation on the left of the above formula, the unknown parameters can be solved directly by inversion, which can not meet the requirements in terms of memory overhead and solution efficiency. This paper adopts the strategy of elimination modification method equation to solve the adjustment. Considering that the dimension of the object coordinate t of the connection point is usually much higher than the image additional parameter x, the connection point coordinate t can be eliminated first and the modification method equation containing only the additional model parameter x can be constructed, as shown in the following formula: After the modified equation is listed, the additional parameters X of each scene image can be solved: After calculating the additional parameters of each scene image, combined with the initial RPC model, the functional relationship between the image coordinates of the tie point and the object ground coordinates can be determined, as shown in the formula. The object coordinate t can be calculated by the forward intersection method from the multi scene images where the tie point is located.
This section describes the robust estimation method of block adjustment parameters under complex ill conditioned conditions based on adaptive weight determination of virtual control points and connection points and the efficient search method of gradient heuristic optimal solution based on sparse matrix, so as to realize the efficient and robust solution of spatiotemporal super largescale block adjustment model parameters.

Automatic diagnosis of ill-condition position
In the adjustment process, due to the influence of gross errors and weak intersection, there may be abnormal areas with abnormal geometric accuracy in the triangulation model after adjustment. These abnormal areas must be located and adjusted again by adding images in the area and manually screening the gross errors of mismatching. For the adjustment of large-scale triangulation model containing massive image data, it is obviously not feasible to manually detect the control points arranged in the field.
In this paper, the internal geometric accuracy evaluation model of the regional network is constructed, and then the geometric accuracy evaluation points are evenly arranged according to a certain grid spacing within the survey area. Based on the evaluation model, the geometric accuracy evaluation indexes at each evaluation point are calculated. Finally, the kernel function model is used to fit the geometric accuracy evaluation indexes of all evaluation points dynamically and adaptively. By judging the fitting error of each point from the two levels of global statistics and local statistics, the points with obvious abnormal fitting accuracy are selected, and the grid area where the point is located is diagnosed as ill conditioned area. According to the block adjustment model established above, the observation error equations are constructed for all tie points and control points respectively and written in matrix form: Where,   The theoretical accuracy of block adjustment is evaluated by taking the unknown covariance matrix obtained by adjustment as a measure (Wang M et al.,2016, Yang Bo et al.,2017. In this paper, the weighted iterative method is used for gross error detection. At the beginning of adjustment, the conventional least square method is still used. Then after each adjustment, the weight of each observation value in the next iterative calculation is calculated according to the residual and other relevant parameters, which is included in the adjustment calculation. If the weight function is properly selected and when the gross error can be located, the weight of the observation value with gross error becomes smaller and smaller until it tends to zero. At the end of the iteration, the corresponding residual will directly indicate the value of the residual, while the adjustment result is not affected by the gross error. For the selection of weight function, the general rule is to use two different functions. At the beginning of the iteration, the weight function should be steeper, while in subsequent iterations, a gentle function is required.
In order to evaluate the adjustment accuracy of the triangulation model and diagnose its ill conditioned area, the geometric accuracy evaluation points need to be evenly arranged according to a certain grid spacing within the survey area, and the geometric accuracy evaluation indexes at each evaluation point are calculated based on the above error evaluation model. Finally, the kernel function model is used for dynamic adaptive sliding window fitting of the geometric accuracy evaluation indexes of all evaluation points. By judging the fitting error of each point from the two levels of global statistics and local statistics, the points with obvious abnormal fitting accuracy are selected, and the grid area where the point is located is diagnosed as ill conditioned area.

Main technical flow of test
Based on the image data and reference data, The main technical roadmap was designed according to the joint adjustment method of multi-source satellite images, which is shown in the figure below.
The geometric stability of the ZY-3 stereo mapping satellite image is good, while the non-stereo satellite image has the phenomenon of weak intersection, which makes the small disturbance in the image space will cause great errors in the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France elevation direction, and the object coordinates in the elevation direction are difficult to converge in the block adjustment. In order to solve the problem of low precision of non-stereo satellite image, the stereo image is used to provide better geometric information for block adjustment, and the plane image is used to provide higher resolution for block adjustment. The joint adjustment model of the two image data is proposed. Through the joint processing, the high-precision mapping of large areas can be realized. The high-precision geometric collaborative processing of stereo satellite images and non-stereo satellite images has the advantages of unified model, high-precision.
In the project, firstly, the obtained multi-source high-resolution images are Preferred according to coverage, overlap, spectral information and geometric accuracy, so as to improve the efficiency of construction of block adjustment.

Accuracy evaluation of reference triangulation
The accuracy of reference triangulation, which was constructed with ZY3 stereo satellite images, is directly related to the quality of large-scale hybrid triangulation adjustment. This section is mainly about verification of the reference triangulation. A certain number of high-precision checkpoints was selected which is evenly distributed on the triangulation model (showned in figure3). The accuracy of the reference triangulation was evaluated with these checkpoints (see in table1). RMSE_LON

Result accuracy analysis
In this paper, Qinghai, west of China, is selected as the experimental area. The satellite images include 256 GF2 satellite images, 125 BJ2 satellite images and 120 ZY3 satellite images. In which，GF2 and BJ2 are non stereo satellite images with a ground resolution of 0.8m, ZY3 is a three-line array stereo satellite image with a ground resolution of 2.1m. The ZY3 satellite image is used to build the reference triangulation network, and other satellite images are transferred to the reference network. After the tie point optimization processing, the overall triangulation processing is completed. 38 checkpoints are evenly distributed in the above test area (see the figure4 for the distribution). The positioning accuracy of processed satellite images is shown in the table below，in which, the positioning accuracy of satellite images before and after processing was listed, including the mean square error and maximum error of checkpoints. RMSE_LON and RMSE_LAT represents the mean square error in longitude direction and latitude direction respectively. As such, MaxErr_LON and MaxErr_LAT represents the maximum error in longitude direction and latitude direction respectively.  It can be found from the table that the adjustment of hybrid triangulation can effectively improve the geometric positioning accuracy of non stereo satellite images, such as BJ2 and GF2 high-resolution satellite images, and also improve the stability of the block-adjustment.
Use the high-precision geometric positioning parameters and DEM obtained from large-scale hybrid triangulation adjustment to produce orthophoto with the the satellite image. This paper describes the large-scale block adjustment technology and experiment of hybrid aerial triangulation with highresolution stereo satellite image and non-stereo satellite image. This technology solves the problem of unstable block adjustment of non-stereo satellite image with weak intersection, and verifies the application of Multi-source Satellite Image hybrid adjustment technology based on stereo image in Orthophoto Image production. The result can be used for high-precision geometric processing of satellite images in difficult areas without ground control points.