Investigation of Optical Flow Techniques for Extracting Non-Rigid Sea Surface Currents

This paper is about performance evaluation of two optical flow (OF) methods for extracting sea surface curved motions like eddies. By incorporating a simple matrix and its rotation in programming procedure, curved motion can be obtained. These two OF methods are Horn/Schunk and Lucas/Kanade. The Horn/Schunk method depends on a smoothness parameter (α) and when it changes, smoothness and reality change too. The Lucas/Kanade method is more complex than previous one. It depends on two parameters, smoothness parameter (Sigma) and window size (Win). Different values for Win and Sigma change smoothness and reality of the flows. Sea surface currents are extracted from two sequential sea surface temperature (SST) images by using OF methods. By using these methods and selecting the appropriate parameters like smoothness (for Horn/Schunk) and window size and smoothness (for Lucas/Kanade) extracting real flows or smooth flows are possible and investigated in this paper.


INTRODUCTION
Critical issues in extracting and monitoring sea surface currents (SSC) are curved motions. These curved motions happen in coasts and other places in sea and depend on some physical characterizes like upwelling. Optical flow (OF) techniques work precisely when motions are rigid directions (like jet currents), but if they move like eddies and have rotating scheme, optical flow techniques may extract the wrong currents. There are many applications in using optical flows (Barron et al., 1994). Sometimes extracted flows should be dense and sometimes they should not be dense and only be accurate. Accuracy relates to reality and density or smoothness relates to appearance. On the other hands, tracing sea surface currents is possible by tracking pattern of sea temperature in successive images. Recently, this technique is widely used by researcher for purpose of sea current extraction. This paper tries to evaluate the performance of two OF techniques by incorporating variety of parameters and settings. Two sequential sea surface temperature images extracted from MODIS sensor used as input of these OF techniques.

OPTICAL FLOW TECHNIQUES
The purpose of using optical flow is approximate calculation of movement in two dimensions. This technique has many usages in different fields (Barron et al., 1994). Conditions and assumptions in OF cause to extract these movements from sequential images. The relation and equation of changing brightness in every point of image and it specifies movement of brightness patterns; is made by these assumptions. If brightness in image coordinate (x,y) at time t is shown with I(x,y,t) and is assumed this pattern is constant, then we have: (1)

Horn/Schunck
The Horn/Schunck method of estimating OF is a global method which introduces a global constraint of smoothness to solve the aperture problem. Here, Horn/Schunck method is used for extracting optical flow. Calculation of first order partial derivatives makes this algorithm so complex. Also computing Laplace equations have heavy calculations in each iteration too.
There is a smoothness term in solution of OF in this method that is suitable for extracting SSCs. Equation of flow that is represented by E, is like a global energy functional which is the sought to be minimized.
E=∬[ (I x u + I y v + I t ) 2 + α 2 (‖∇u‖ 2 + ‖∇v‖ 2 ) ] dx dy Where Ix, Iy and It are the derivatives of image intensity values along x, y and time dimensions respectively, u and v are OF vector and the parameter α is regularization constant. Larger values of α lead to a smoother flow (Horn and Schunck, 1981).

Properties
Advantages of the Horn/Schunck algorithm include that it yields a high density of flow vectors, i.e. the flow information missing in inner parts of homogeneous objects is filled in from the motion boundaries. On the negative side, it is more sensitive to noise than local methods (Horn and Schunck, 1981).

Lucas/ Kanade
In computer vision, Lucas/Kanade assumes that the flow is essentially constant in a local neighbourhood of the pixel under consideration, and provides flows for all the pixels in that neighborhood, by the least square criterion. By combining information from several nearby pixels, this method can often resolve the inherent ambiguity of the OF equation (Lucas and Kanade, 1981). It is also less sensitive to image noise than point-wise methods (Lucas and Kanade, 1981). On the other hand, since it is a purely local method, it cannot provide flow information in the interior of uniform regions of the image (Lucas and Kanade, 1981). This method The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-2/W3, 2014 The 1st ISPRS International Conference on Geospatial Information Research, 15-17 November 2014, Tehran, Iran assumes displacement of the image contents between two nearby instants (successive images) is small and approximately constant within a neighbourhood of the any given points. The least squares approach implicitly assumes that the errors in the image data have a Gaussian distribution with zero mean (Lucas and Kanade, 1981). The Lucas/Kanade method can be used only when the image flow between the two images is small enough for the differential equation of the OF to hold, which is often less than the pixel spacing (Lucas and Kanade, 1981).

STUDY AREA AND DATA SETS
This section introduces the study area and also data sets that used for evaluating OF techniques.

Caspian Sea
The Caspian Sea is the largest lake on our planet. It is bigger than the Great American lakes and Lake Victoria in Africa by surface area. Ecosystems in the Caspian Sea region have been heavily modified by anthropogenic activities, mainly as a result of changes in the water flow and degradation of the water quality in the ecosystems (Zonn et al., 2010). Changes in the regional environment have influenced regional economies, particularly obvious in the impacts on fish stocks. Some reports focus on the root causes for the most important drivers of environmental change in the region: habitat and community modification (Zonn et al., 2010).

Sea Surface Temperature
Brightness and digital number of point in images is SST, then OF can track temperature patterns. This method leads us to recognition of movements of surface temperature. That means by tracing surface temperature catching surface current is possible. For extracting SSC from SST images, It needs one or more pair/successive images. The SST images calculated from MODIS sensor that is boarded on Terra and Aqua satellites. Aqua is ascending orbit satellite and is known to afternoon satellite. Terra is descending orbit satellite and is known to morning satellite. The revisit cycle of these satellites is 12 hours (Ghalenoei, 2014). Time distance of an image without cloud covering from Aqua to Terra or inverse is almost 6 hours. Then finding these images with minimum time distance are very appropriate for extracting SSC but this is very difficult practically because Caspian Sea covers with clouds in almost days of year and this causes SST images useless for tracing temperature pattern. Here, Fig 1 shows two SST images (Ghalenoei, 2014). After getting SST images, preparing successive images as input of OF algorithms is necessary. In this regards, the two images with minimum time distance are a pair images. In other word, if the SST images are sequential due to time, first and second images are first pair images and second and third images are second pair images and so on. Here two SST images make one pair image (Ghalenoei, 2014).

EXPRIMENT AND RESULTS
This section synthetically produces shape and rotates it like eddy current to show robustness and performance of utilized method.

Rotating a matrix
There is a matrix A due to first image. In Matlab language programing, it can be made as following: A=zeros(50,50); A(16:35,16:35)=1; In this matrix all of pixels are 0 except 16 th to 35 th of rows and columns that are 1. Rotating matrix A by 5 degree in a counter clockwise direction around its center point is possible by using imrotate function (Fig 3). Rotated matrix (matrix B) is like an eddy on sea surface that OF wants to extracted it. First, Horn/Schunck method is used and is analysed its smoothness parameter (α) when it changes from 0.1 to upper values.

Results of Horn/Schunck method
The results of Horn/Schunck method are illustrated when smoothness parameter (α) changes, the flows may lose the rotated scheme. It depends on α when it's less than 1.0, the flows can show rotating but when it's more than 1.0, the flows mistake to extracting its rotating center because of it wants to make more smooth flows and doesn't pay attention to the purpose that it must be a rotational motion (Figure 4-9) .

Results of Lucas/Kanade method
This method depends on two parameters; Win and Sigma are respectively window radius (for integration) and amount of smoothness. Intuitively, a small integration window would be preferable in order not to "smooth out" the details contained in the images. But if Win is constant, smoothness of results can be changed by Sigma (a parameter that controls amount of smoothness). Results show investigation of Win parameter for values 2 and 5 and for each value (2 or 5) Sigma parameter is changed to see difference of them (Figures 10-21). Whatever Win parameter is larger, the flows are more smooth and better for extracting rotating scheme (like sea surface eddies) but should pay attention to Sigma, use of less value for Sigma (like 0.5 or 1) and large value for Win (like 5) results appropriate schemes for monitoring the eddies. Thereupon figure 17 and its parameters have the best performance for extracting rotational movements.  For smoothness condition in α=3 (Fig 24), some of curved motions are ignored relative to α=0.1 (Fig 22) but the flows in Figure 24 are more continuous and smooth. The results of Lucas/Kanade are more smooth when Win increases and by decreasing Sigma (Sigma=1) it can obtain real flows (Fig 26), but in Figure 27 when Sigma increases (Sigma=3), the reality decreases and the extracted flows are more smooth. It suggests for sea surface currents that are like jet currents and when they move like curved lines it's better to decrease Sigma. Each vector for example in figure 23 has different direction with its match ( Figure 24) and when α increases, the norm of vectors increase too regards to size of vectors. Using arccos function for calculating angular differences can provide a map that shows the most disturbances for extracted directions.

Extracting SSCs from SST
= cos −1 ( Where, A and B are respectively first and second vectors. ∥ ∥ and ∥ ∥ are respectively norm of A and B.

CONCLUSION
Using small values for smoothness parameter (α) in Horn/Schunk method obtains real schemes of motions, but if α increases, the flows will get smooth and the smoothness condition ignores some of real motions (like eddies). Using The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-2/W3, 2014 The 1st ISPRS International Conference on Geospatial Information Research, 15-17 November 2014, Tehran, Iran small window size in Lucas/Kanade method is preferable but it's less smooth relation to great size. The norms of vectors increase when α (in Horn/Schunck) and Sigma (in Lucas/Kanade) get great values. Reality and smoothness are two main characterizes that they change together. When smoothness increases, reality decreases and vice versa. Angular differences maps (Fig 28 and 29) show the errors that occur when two different smoothness parameters are chosen, also in high differences; it shows the main places that optical flow techniques are unable to determine real motions.