DETECTION AND CORRECTION OF JITTER EFEFCT FOR SATELLITE TDICCD IMAGERY

For the on-orbit high resolution satellites, attitude jitter is a common and complex phenomenon which may deteriorate the image quality and mapping accuracy. In this paper, an image-based jitter inversion and estimation model is derived considering time delay integration (TDI) effect, and the attitude is then updated by taking into account the estimated jitter information. The imagery is finally resampled through virtual re-imaging using the updated accurate attitude to improve the image geometric quality. The proposed method is validated by experiments using the Chinese Ziyuan-3 (ZY-3) satellite imagery.  Corresponding author: Shijie Liu (Email: liusjtj@tongji.edu.cn)


INTRODUCTION
High-resolution optical remote sensing satellites are extremely important in both the civil and military sectors. These satellites have been highly valued by countries all over the world and have promoted the continuous development of high-resolution remote sensing technology. However, attitude jitter restricts the geometric quality and imaging performance of highresolution optical satellites especially when using Time Delay and Integration (TDI) (Ayoub et al.,2008). If the jitter is not detected and compensated timely and effectively, it will lead to a decline in remote sensing image quality and affects the accuracy of image registration and change detection (Ayoub et al.,2008), image mosaicking (Jiang et al.,2017), and elevation modelling (Girod et al.,2017).
Many scholars have detected and analysed the jitter of highresolution optical satellites. Detection methods can be classified into: indirect detection methods based on a non-attitude sensor and direct detection methods based on an attitude sensor (Tang et al.,2020). The indirect methods rely on star camera, surveying and mapping products, or imageries, etc. Taking the stars as the reference, in case of micro-vibrations, the position of the star in the image will show obvious periodicity. Amberg et al (2013) extracted the track of micro-vibrations in across track direction from the position of the star in columns for each line based on image Fourier Transform. This technique has proved to be efficient and led to accurately assessing attitude vibrations but it can access only jitter in across track direction. With the reference Digital Orthophoto Map (DOM), Liu et al (2016) proposed jitter detection method based on imageries and dense ground control points, the discrepancies between the projected image points and the measured ones reflect the influence of attitude jitter. The results demonstrated the existence of 0.65Hz attitude jitter with amplitude of up to 2.63 pixels with Chinese ZiYuan-3 (ZY-3) satellite, but the universality of this method is relatively poor. Based on multitemporal image correlation, CCD to CCD or band to band correlation, obvious jitter was detected in the Terra Satellite Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) (Teshima and Iwasaki,2008), the High Resolution Imaging Science Experiment (HiRISE) camera (Mattson et al.,2009;Sutton et al.,2017), the Lunar Reconnaissance Orbiter Narrow Angle Camera (LROC-NAC) (Mattson et al.,2010), ZY-3 (Tong et al.,2015a(Tong et al., , 2015b(Tong et al., ,2017a(Tong et al., ,2017bPan et al.,2017;) and Mapping Satellite-1(Sun et al, 2015). This method is highly precise and easy to implement.
The direct detection methods are based on high precision attitude sensors, such as star sensor, gyroscope or angular displacement tracker. As for the star sensor data of ZY-3,  proposed a penalized spline method for establishing an attitude model. They modelled the attitude jitter with piecewise and continuously differentiable polynomials and the attitude noise was smoothed out with a penalty function. The penalty spline model can limit the registration error less than 0.1 pixel. Mo et al (2017) obtained the platform jitter with a frequency range of 0.64~0.71 Hz by analysing the ZY-3 sensor data. For the GaoFen-9 satellite, Zhang et al (2018) found obvious jitter with amplitude 0.15 rad/s 2 at a frequency of 100 Hz, then they corrected the attitude data by combining the star sensor attitude observations and micro-vibration accelerometer measurements, the correction effect was assessed with road straightness and the jitter amplitude decreased from 0.9 pixels to 0.4 pixels.
If the frequency of satellite attitude jitter is higher than the attitude measurement sampling rate, the geometry relationship of ground points and corresponding image points cannot be accurately described by the imaging model (Liu et al.,2016). And limited by sampling frequency and some amount of noise or drift in the Inertial Measurement Units (IMU) data, attitude reconstruction based on star trackers and gyro is not a reliable source of information for correcting jitter (Sutton et al.,2017). Therefore, we need to recover the real attitude information based on other information. And in the process of the multistage integration imaging of Time Delay Integration Charge Coupled Devices (TDICCD), image offset caused by jitter of each stage changes with time, the impact on the synthesized image is no longer the same as that of single-stage integration image. And if we ignore the multistage integration process, the errors of the jitter information obtained from detection and estimation will increase with the increase of integral series and frequency.
In this paper, we construct an image-based jitter inversion and estimation model considering TDI effect based on the imaging principle of TDICCD to solve the jitter characteristics, and then we update the attitude by taking into account the detected jitter information, and finally we eliminate the geometric distortion in the imageries based on reimaging. In this case, implementation of the jitter detection and correction are performed on Chinese ZY-3 multispectral imageries. It is found that this method can effectively eliminate the geometric distortion caused by jitter.

Jitter inversion and estimation
TDICCD is an area array CCD. During imaging, multiple rows of CCD linear array expose the ground objects in turn along the track direction, and after one exposure, the charge is be transferred to the next row until multi-level imaging is completed. All generated charges are transferred to the readout register and then the image of this row is read out (Chen et al.,2002). The schematic diagram of TDICCD integration under jitter effect in across-track direction is shown in the In the multi-level integration process of TDICCD, the image offsets caused by platform jitter in each stage of integration change with time, so the integrated image is the synthesis of multi-level jitter image. Assuming that the imaging starts from time t, the waveform of platform jitter is expressed as a simple sine wave (Schwind et al.,2009).
Then we can express the jitter after TDI integration as： A , f and  are the amplitude, frequency and initial phase of jitter， i t is the initial integration time of line i.
According to the characteristics of TDICCD linear array push broom sensor with parallax observation, when the attitude of the satellite is stable, the parallax of each line is a constant, which is determined by the physical distance, orbital altitude and flight speed of the sensor. When the attitude of the satellite is affected by jitter, there is geometric displacement in each image line, the parallax of each line is no longer constant. s (Teshima and Iwasaki,2008).

Subpixel dense matching based on SVD-RANSAC
To detect and estimate attitude jitter from imagery, a pixel-bypixel dense matching scheme based on the subpixel phase correlation method using singular value decomposition and unified random sample consensus (SVD-RANSAC) (Tong et al., 2015c) is adopted, which involves three major steps: (1) The two images with fixed imaging time interval are tagged as image M and image S. The Wallis filter is used to enhance the image and improve the image contrast. The image is translated according to the known imaging time interval to achieve initial registration.
(2) Then, the sub-pixel offset between each window is obtained by carrying out the SVD-RANSC sub-pixel phase correlation method. SVD-RANSAC can estimates subpixel translational with a high reliability and strong robustness than Hoge's method and the unified RANSC algorithm (Tong et al.,2015c).
(3) Before the subsequent analysis, it is necessary to eliminate mismatches and reduce the influence of noise.
(1) we calculated the normalized cross-correlation (NCC) coefficients between the matching windows, and we removed the matching points whose correlation coefficients are less than the threshold 0.7. (2) According to the principle of the triple mean squared error of parallax of the matching point pair in the same image line, we eliminated the matching points with large offset.
After obtaining the relative error between parallax images, we can get the jitter amplitude and frequency according to the quantitative relationship between the jitter displacement and parallax for the following analysis.

Jitter correction
According to section 2.1, image distortions in the cross-track and along-track directions are accordingly obtained. In general, for nadir or close-to-nadir view, the cross-track distortion () across jt is mainly caused by the variations of roll angle, and the along-track distortion () along jt is mainly caused by the variation of pitch angle (Liu et al.,2021). The roll and pitch angle variations can be calculated from the corresponding image distortions by the sensor parameters as (Ye et al.,2020).
The rotation matrix si A represents the attitude obtained by the traditional satellite sensor and gyro, which does not record the high frequency jitter information.
Where  represents the pitch angle,  represents the roll angle，  represents the yaw angle.
Taking into account the estimated jitter information, we can get the real attitude containing jitter, which is expressed as follows: I is the identity matrix， []  is the cross product of  . Since  is relatively small, the higher-order term can be ignored, and the real attitude can be expressed by the following small angle equation.
After updating the attitude, we carried out image correction based on reimaging to obtain the corrected image , Figure 2 shows the scheme of image jitter correction.
(1) First, we calculated the object coordinates ( ) ,, , rc by the forward projection on the elevation surface H using the jitter-free attitude data. H is average height.

Experimental data
In order to assess the effect of jitter estimation and compensation, we carried out the experiment with three CCDs of ZY-3 multispectral imageries, and per CCD image consists of four bands (Tang et al., 2015). In the experiment, quadratic polynomial was performed to smooth the attitude observation to obtain the jitter-free attitude data. The band 1 and band 2 multispectral imageries are used to obtain jitter information.

Comparison of attitude results
The yaw angle has little influence on imaging geometry (Liu et al., 2021), so we only consider roll and pitch angle. According to section 2.2, we updated the attitude by taking into account the detected jitter information. In order to verify the correctness of the algorithm, we compare it with the original attitude of ZY-3, and the results are shown in the Figures 4-Figures 6, we can find that the results are coincide with the original attitude data of different CCDs.

Jitter compensation analysis
After generating corrected image according to section 2.3, we perform dense image matching again to check whether the jitter is effectively compensated. Figure 7, Figure 8 and Figure 9 show the parallax before and after jitter compensation, we can see clearly that the periodic disappears and the parallax is concentrated near 0. And the root mean square error in crosstrack direction drop from 0.3 to 0.1 pixels and the root mean square error in along-track direction drops from 0.2 to 0.1 pixels after effectively correcting the jitter distortions. The above results show the effectiveness of our method. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2022 (a) (b) Figure 9. CCD3 parallax of B1and B2 before and after jitter compensation ((a) is the along-track direction parallax (b) is the cross-track direction parallax)

CONCLUSIONS
In this study, satellite jitter inversion and estimation model based on imaging principle of TDICCD cameras were derived, and the attitude was then updated by taking into account the estimated jitter information. Meanwhile, image correcting was performed by re-imaging using the updated accurate attitude. Experiments conducted using image data and attitude data of ZY-3 satellite indicate that the proposed method achieves well performance of jitter detection and compensation.

ACKNOWLEDGMENT
This work was supported by National Natural Science Foundation of China (42171432, 41771483), Shanghai Science and Technology Project (21511103800), and Fundamental Research Funds for the Central Universities. The experimental ZY-3 data was provided by Chinese Satellite Surveying and Mapping Application Center (now Land Satellite Remote Sensing Application Center).