AUTOMATED MOSAICKING OF GEOSTATIONARY OCEAN COLOR IMAGER BY COMBINATION OF SPATIAL AND FREQUENCY MATCHING

In general, image mosaicking is a useful and important processing for handling images with narrow field of view. It is being used widely for images from commercial cameras as well as from aerial and satellite cameras. For mosaicking images with geometric distortion, geometric correction of each image should be performed before combining images. However, automated mosaicking images with geometric distortion is not a trivial task. The goal of this paper is the development of automated mosaicking techniques applicable to handle GOCI images. In this paper, we try to extract tie-points by using spatial domain and frequency domain matching and perform the mosaicking of GOCI. The method includes five steps. First, we classify GOCI image slots according to the existence of shorelines by spatial domain matching. Second, we perform precise geometric correction on the slots with shorelines. Third, we perform initial sensor modelling for the slots without shorelines and apply geometric correction based on the initial model. Fourth, the relative relationship between the slots without shorelines and the slots with shorelines is estimated through frequency domain matching. Lastly, mosaicking of geometrically corrected all 16 image slots is performed. The proposed method was verified by applying to real GOCI images. The proposed method was able to perform automated mosaicking even for images without shorelines, and its accuracy and processing time were satisfactory. For future research, we will improve frequency matching to generate multiple tie-points and to analyse the applicability of precise sensor modelling directly from frequency matching. It is expected that the proposed method can be applied to the follow-up sensor of the GOCI, GOCI-II, and other ocean satellite images. * Corresponding author


INTRODUCTION
In general, image mosaicking is a useful and important processing for handling images with narrow field of view.It is being used widely for images from commercial cameras as well as from aerial and satellite cameras.For mosaicking images with geometric distortion, geometric correction for each image should be performed before combining images.However, automated mosaicking images with geometric distortion is not a trivial task.In this paper, we aim to mosaic images from the GOCI (Geostationary Ocean Color Imager), which has been installed on the geostationary satellite, GK-1, and takes images over the area of 2,500km x 2,500km through consecutive 16 slots.Each slot contains significant distortions due to earth curvature, variation of satellite position and attitudes, etc.One of the most important and mission critical task is automated combination of the 16 image slots into one image without geometric discrepancy among them.Many studies have been carried out on the acquisition of spatial information through geometric correction and on the compensation of geometric distortions of various satellite and aerial images.Mostly, geometric correction for terrestrial satellite images is performed by using GCPs (Ground Control Points).For geostationary ocean color imager, a similar process called image navigation has been applied (Lee et al., 2005).Small image chips of shorelines with known coordinates are collected beforehand and they are matched against ocean images.However, images from ocean observing satellites such as GOCI often have slots with no shorelines or cloudy shorelines.In this case, we cannot use the geometric correction technique through image navigation.The goal of this paper is the development of automated mosaicking techniques applicable to handle GOCI images.In particular, our research aim to handle this problematic case.We propose to handle slots without shorelines by estimating relative orientation between slots without shorelines and slots with shorelines and to perform geometric correction and mosaicking based on the relative orientation.Estimation of relative orientation in our case is not straight forward since it usually requires tie-point extraction (Yamamoto et al., 2007;Herbert et al., 2007;Rublee et al., 2011).In general, for estimation of relative orientation between adjacent images extracted tie-points based on feature points are used.Ocean image slots without shorelines do not have enough features to be used as tie-points.In this paper, we try to extract tie-points by using frequency domain matching as an alternative method and perform the mosaicking of the ocean satellite images.

PROPOSED METHOD
For mosaicking multiple images,we must perform accurate and robust geometric correction of individual images with respect to the reference plane (Kim et al.,2017).In case of ocean satellite images, we can not apply general feature matching method.Because the texture information is not sufficient.Figure 1 shows the example of feature matching failure of ocean images.
Therefore, we solve this problem by applying frequency domain matching.In this chapter, we introduce frequency domain matching used and explain our method.

Frequency Domain Matching
Frequency domain matching is a technique to extract tie-points by interpreting images in the frequency domain, applying phase correlation and converting a correlation peak in the frequency domain onto the spatial domain.To apply frequency domain matching for images with insufficient texture, we applied a Laplacian filter onto image before frequency transformation (Kim et al., 2016).Through this, success rate of frequency domain matching can be increased to a great deal.Figure 2 shows the process of frequency domain matching.
In our earlier research (Kim et al., 2016), we showed that frequency domain matching had a 99% of success rate in 192 GOCI image pairs.Figure 3 shows the example of frequency matching of ocean images.

Mosaicking of Geostationary Ocean Color Imager
Overall processing for GOCI mosaicking in this paper includes five steps.Figure 4 shows the process of the proposed mosaicking method.
In the first step, we classify GOCI image slots according to the existence of shorelines within the slot.Slots with shorelines undergo spatial domain matching and slots without shorelines undergo frequency matching.In the second step, we perform precise geometric correction on the slots with shorelines.Precise geometric correction step includes spatial domain matching between each slot and shoreline, precise sensor modelling and image mosaicking.In this step, matching is performed by applying sobel edge detection to the slots and applying grey level correlation.Precise orientation of the slots are estimated rigorously by precise sensor modelling using the matching results.We apply a sensor model for the navigation system of GOCI. Figure 5 shows the example of precise sensor modelling result.In the third step, we perform initial sensor modelling for the slots without shorelines and apply geometric correction based on the initial model.Initial sensor modelling is performed by the initial metadata and the sensor model of GOCI.After this step, we assume that the geometric correction for rotation and scale has been completed, and that, we only need to estimate further for translation.Figure 6 shows the example of initial sensor modelling result for slots without shorelines.
Figure 6.Initial sensor modelling of images not including shoreline In the fourth step, the relative relationship between the slots without shorelines and the slots with shorelines is estimated through frequency domain matching.Through this step, we can calculate translation parameters between the slots.Figure 7 shows the example of frequency matching result.Lastly, mosaicking of geometrically corrected all 16 image slots is performed.In this step, we perform the work of attaching the slots with shorelines and the slots without shorelines.Through these five steps, we can produce mosaicked image of GOCI by proposed method.
Figure 8. Mosaicking result by using proposed method

Experiment Data
For verifying the proposed method, we used eight GOCI L1A data sets.The data sets were photographed at every hour from 9:00 am to 3:00 pm on April 5, 2011.They cover the area of 2,500 x 2,500km width including Republic of Korea, Japan, East coast of China and northern part of Taiwan.The datasets corresponding to each time were composed of 16 slots.Each slot was photographed in 8 bands with 500m spatial resolution.
Figure 9 shows the experiment data photographed at 12:00pm.
The number on the upper left corner of the images means the slot number.

Experiment results
For verifying the proposed method, we performed two experiments.The objective of the first experiment to check whether or not the proposed method can be useful for mosaicking GOCI images.The objective of the second experiment is to perform the accuracy verification by using GCPs by applying the proposed method to the images with shorelines.
For the first experiment, we classified the images into two groups.The first group consisted of slots without shorelines in the image.The 12 th , 13 th , and 14 th slots belonged to this group.
The second group consisted of slots with shorelines in the image.All slots except those in the first group belong to this group.We applied frequency domain matching to slots in the first group which do not include shorelines with adjacent slot of each slots.For the second group, we applied precise geometric correction through shoreline matching.After all of our processing steps, we generated a mosaic image such as figure 10.
Figure 10 shows the mosaicked image with shorelines in yellow line.
For the second experiment, we chose 5 th , 7 th , and 9 th slots as the first group and all other slots as the second group.After all of our processing steps, we generated a mosaicked image shown in figure 11.From visual inspection, there is no significant difference between the two results.

Accuracy analysis
For verifying the accuracy, we generated 52 GCPs for 13 slots with shorelines from the Google map.We calculated RMSE(Root Mean Square Error) between true coordinates of GCP and estimated coordinates of the corresponding point.
Table 1 shows the results.We can observe that there are larger errors in the group using frequency matching.This result was expectable since frequency matching estimated translation only from a single match result whereas shoreline matching estimated the full exterior orientations from multiple match results.Note that frequency matching was meant to be used when there is no shorelines.In table 2, we compared the accuracy of frequency matching with those of shoreline matching and of the model of previous slot.Note that currently slots without shorelines are being processed using the sensor models of the slots photographed previous time.
Compared to the current conversion, frequency matching achieved a great accuracy improvement.We can observe that if GCP is present and can perform precise sensor modelling, we can have the best accuracy.However, in case of not using it, we can confirm that the method used by GOCI is less accurate than the proposed method.We have analysed the time required for frequency matching on Intel (R) Core (TM) i5-6600 CPU @ 3.30GHz, RAM 16GB, and NVIDIA GeForce GTX950 graphics card.An average of 0.10 seconds per slot was required for frequency domain matching.This means that the method is suitable for mosaicking of GOCI images.

CONCLUSIONS
We proposed a method to perform automated mosaicking even for images without shorelines, and showed that its accuracy and processing time were satisfactory.For future research, we will improve frequency matching to generate multiple tie-points and to analyse the applicability of precise sensor modelling directly from frequency matching.It is expected that the proposed method can be applied to the follow-up sensor of the GOCI, GOCI-II, and other ocean satellite images.

Figure 4 .
Figure 4. Process of proposed method

Figure 7 .
Figure 7.Estimated translation parameters through frequency matching

Table 1 .
RMSE analysis of second experiment result