FREQUENCY FEATURES FOR DETECTING EVENTS IN VIDEO SEQUENCE OF VIDEO-EEG MONITORING DATA

The work is devoted to the study of the frequency features of the optical flow obtained from the video record of long-term videoelectroencephalographic (video-EEG) monitoring data of patients with epilepsy. It is necessary to obtain features to recognize epileptic seizures and differentiate them from non-epileptic events. We propose to analyze the periodograms of the smoothed optical flow calculated from the fragments of the patient's video recordings. We use Welch's method to obtain periodograms. The values of the power spectral density of the optical flow at the selected frequencies will be used as features. Using the clustering algorithm, four groups of events were identified in video recordings. * Corresponding author


INTRODUCTION
In clinical practice, video-electroencephalographic monitoring, a method of long-term synchronous recording of an electroencephalogram (EEG) and video image, has become widely used. Simultaneous video recording of the patient's clinical state and the bioelectrical activity of the brain makes it possible to reliably diagnose epileptic seizures and differentiate them from non-epileptic events (Patel, 2013;Chen, 2016). For recording and visual analysis of video-EEG data, specialized software is used, for example, Galileo NT Line package, which has a set of functions for signal processing and analysis, as well as statistical data analysis. If diagnostically important fragments of the EEG are found, the physician needs to revise the area of interest in the video recording for visual assessment and differentiation of an epileptic and artifact event. Visual analysis of video data is extremely laborious; therefore, it becomes necessary to develop methods for automatic registration of epileptic seizures from video sequences obtained during video-EEG monitoring.
Some works on the analysis of video recordings for the detection and recognition of epileptic events are known. The work (Karayiannis, 2001) presents methods for measuring the motion strength and motor activity of newborns using video recording. The quantitative characteristics obtained in the form of signals are used to differentiate myoclonic and clonic seizures and to distinguish seizures from the normal behavior of the newborn. The motion strength is defined as the area of the moving parts of the infant's body. To outline such fragments, the wavelet transform of frames, median filtering, and segmentation operations using an adaptive version of the kmeans algorithm are used. The change in time of the coordinates of characteristic points, selected automatically on the child's limbs and tracked using the KLT (Kanade-Lucas-Tomasi) algorithm, generates signals that characterize motor activity. Modifications of methods for measuring motion strength and motor activity are presented in the works (Karayiannis, 2003;Karayiannis, 2005).
In the works (Kouamou Ntonfo, 2012;Cattani, 2017) the authors solved the problem of real-time detection of clonic seizures in newborns by the sequence of images obtained from video cameras. The value of the filtered average optical flow calculated as the sum of the binarized pixel-by-pixel difference in luminance of adjacent frames of the video sequence is analyzed. In (Kouamou Ntonfo, 2012), a feature of a seizure is the periodicity of the optical flow, which is detected using a hybrid autocorrelation-YIN estimation technique. In (Cattani, 2017), the Maximum Likelihood criterion is used to determine the periodicity of the optical flow.
The paper (Geertsema, 2018) presents an algorithm for recognizing convulsive seizures in real-time from a sequence of frames recorded by a video camera. To analyze video images and detect events, the method described in (Kalitzin, 2012) is used. The method consists in computing the components of the optical flow associated with group transformations of objects in frames (translation, rotation, dilatation, shear) and band-pass temporal filtering to identify occurrence of clonic movements.
In the works (Geertsema, 2020;van Westrhenen 2020), the authors analyze video sequences recorded by multiple cameras to detect nocturnal motor seizures and central apneas occurring in the aftermath of epileptic seizures. The characteristics of the sigmoid patterns of the time-frequency spectrum (modulation maximum amplitude and total spectral power modulation at the time of the event) in the range of 0.1-1 Hz of translation, dilatation, and shear rates are used as features of a diagnostic event. These velocities are calculated by the method described in (Kalitzin, 2012) from the optical flow generated by the patient's movements.
Previously, the authors of (Murashov, 2019;Murashov, 2019a) proposed to detect events in a video recording by the magnitude of the optical flow, which characterizes the degree of mobility of the frame area in which the patient is located. The algorithm was designed to detect both convulsive and non-convulsive seizures. The tests showed that the detected events quite accurately coincided with the events detected during the analysis of the EEG wavelet spectrograms. In (Murashov, 2020), an algorithm for the synchronous analysis of the EEG signal and video recording for the detection and differentiation of diagnostic and artifact events was proposed. The algorithm combines a threshold detector of brain activity using the ridges of wavelet spectrograms (Obukhov, 2018) and a threshold detector of events in terms of optical flow (Murashov, 2019;Murashov, 2019a). The results of the analysis of clinical data recorded on EBNeuro equipment and Galileo NT Line package software showed the fundamental possibility of reliable distinguishing artifact events from epileptic seizures (Murashov, 2020). However, for more reliable recognition of epileptic seizures, additional features are needed.
The study of publications has shown that frequency analysis of signals obtained from an optical flow is widely used to extract features of epileptic seizures. This work aims to study the frequency features of the optical flow of video recording for the recognition of diagnostic and artifact events during the synchronous analysis of video-EEG data. Unlike well-known works, the proposed study will consider a wider range of events recorded on video. We propose to analyze the periodograms of the smoothed optical flow, calculated from the fragments of the video recording of the patients. Since this work does not consider the multidirectional components of the optical flow, the classical method of spectral analysis of signals will be used to obtain periodograms. As features, we will use the values of the power spectral density (PSD) at the selected frequencies.

COMPUTATION OF OPTICAL FLOW.
Analysis of publications in the domain of detecting seizures from video sequences showed that the most common approach is based on the analysis of optical flow. In works (Murashov, 2019;Murashov, 2019a) it was proposed to detect diagnostic events using the measure, characterizing the degree of mobility of the region of interest. The region of interest is the part of the frame where the patient is located. A measure of the mobility of the region of interest is the total value of the optical flow calculated for each frame of the video sequence:  (1), the Lucas-Kanade algorithm (Lucas, 1981) is applied. As far as the noise component is present in model (1), the smoothed value of the activity criterion   Jn should be used to detect events. The smoothed   Jn value is obtained using a discrete version of the Kalman-Bucy filtering algorithm (Kalman, 1961). We apply the Kalman-Bucy algorithm because it provides the optimal estimate in the sense of minimum error variance. The decision to fix a diagnostic event is made according to the threshold rule. To increase the reliability of event detection, it is necessary to take into account not only the amplitude but also the frequency characteristics of the optical flow. In the next section, we will present a technique for extracting the frequency features of the optical flow.

FREQUENCY FEATURES OF EVENTS
To construct periodograms, the Welch method with a Hamming window with 50% overlap is used (Marple, 1987). The analyzed video data are presented in the form of three-minute fragments of the patient's video recording which fix various events and their combinations: seizures, sleep, movement, food intake. The frame (sample) rate is equal to 20 frames per second. Periodograms are obtained for three window sizes: 10, 200, and 600 samples. Examples of periodograms computed from video records of different events of one of the patients are shown in Figure 1. We studied 24 fragments of video recordings of three patients that fixed the following events: epileptic seizure, intense movement in the frame, sleep, rest or smooth movement after the seizure, food intake, smooth movement. Figure 2 shows graphs of the PSD levels of periodograms of events at different frequencies. Figures 1 and 2 show that different events take different values of the power spectral density. To study the structure of the data that contain the obtained spectral characteristics of the optical flow, we use the technique of cluster analysis. The application of the clustering algorithm to distinguish groups of events by levels of power spectral density of periodograms obtained from video recordings of different events is described in the next section.

CLUSTERING EVENTS IN THE FREQUENCY FEATURE SPACE
To study the possibility of using classifiers for detecting events, a clustering algorithm was applied. The algorithm is based on the search for locally optimal data partitions with automatic selection of the optimal number of clusters (Zhuravlev, 2006). For optimization, the criterion for the minimum of the sum of intraclass variances is used:  The optimal number of clusters k in the range 11 a k b     is determined from the condition for the maximum of the functional The feature space is formed from the PSD values of the periodograms of the optical flow obtained with three window sizes (see Section 3). The PSD values were taken at 14 frequencies in the range from 0.5 to 8.8 Hz. Thus, the dimension of the composited feature vectors is equal to 42. The optimal number of clusters was chosen in the range from 2 to 8.
The results of the clustering showed that there are four groups of events characterized by similar PSD values at the selected frequencies: (a) movement and combination of food intake and movement; (b) seizures and intense movement; (c) food intake and smooth movement, as well as a fragment that recorded the patient's sleep and partially the seizure (d) sleep and smooth movement;.

Figure 2. Graphs of PSD levels of events obtained at different frequencies
The relative distances between the centers of the obtained clusters are given in Table 1, and the values of the standard deviations of the feature descriptions of events The results given in Tables 1 and 2 show that the feature descriptions of most of the considered types of events are fairly well separated and compact in the feature space. The centers of the "Movement" and "Eating" clusters are located relatively close, since food intake is usually accompanied by the movement of the hand, and, therefore, the nature of movements in these groups of events can be similar.
It is possible to increase the accuracy and stability of the partition by reducing the duration of the analyzed video fragments. Differentiation of seizures and movement artifacts will be carried out in the synchronous analysis of video and EEG recordings.

CONCLUSIONS
The preliminary results of the frequency analysis of the optical flow computed from real clinical video-EEG monitoring data are presented. It is shown that the clustering algorithm distinguishes four groups of events according to the level of power spectral density. The results confirm the possibility of applying recognition algorithms to detect events by frequency features. Further research will be focused on the selection, training, and testing of classifiers.