VISUALIZATION OF SPATIO-TEMPORAL RELATIONS IN MOVEMENT EVENT USING MULTIVIEW

Spatio-temporal relations among movement events extracted from temporally varying trajectory data can provide useful information about the evolution of individual or collective movers, as well as their interactions with their spatial and temporal contexts. However, the pure statistical tools commonly used by analysts pose many difficulties, due to the large number of attributes embedded in multiscale and multi-semantic trajectory data. The need for models that operate at multiple scales to search for relations at different locations within time and space, as well as intuitively interpret what these relations mean, also presents challenges. Since analysts do not know where or when these relevant spatio-temporal relations might emerge, these models must compute statistical summaries of multiple attributes at different granularities. In this paper, we propose a multi-view approach to visualize the spatio-temporal relations among movement events. We describe a method for visualizing movement events and spatio-temporal relations that uses multiple displays. A visual interface is presented, and the user can interactively select or filter spatial and temporal extents to guide the knowledge discovery process. We also demonstrate how this approach can help analysts to derive and explain the spatiotemporal relations of movement events from taxi trajectory data.


INTRODUCTION
Individual or collective movement events contain potentially meaningful information about the spatiotemporal evolution of movers, and they also reflect urban transportation patterns (Wang. 2016).Finding and comprehending the relations between movement events, as well as the relations between movement events and elements of the city context, are helpful to the government officials who make decisions about the transportation development of cities, which may in turn improve the daily quality of life within the city (Berenjkoub. 2015).In addition, it is important to understand these spatial-temporal relations in an easy and efficient way.It is hoped to make full use of emerging technologies, and help government or other analysts to work in an intelligent way.
Up to the present time, a number of advanced data acquisition technologies have been developed recently for capturing movement data: location positioning using cell phones and GPS, personalized user-tagged cards for public transportation, and video analysis to assess the flows of people and vehicles.The volume of movement data has increased to an unprecedented degree, the datasets have become very large, and their structures have become very complex.Moreover, the scale and semantics of these datasets cause some challenges in extracting relations from these datasets using traditional methods.Determining the spatiotemporal relations in these multi-scale and multi-semantic movement datasets is especially difficult (Wibisono, 2016).
Facing these challenges, many scientists and researchers are interested in exploring movement data through visual analytics technologies [4] .These methods combine visual interactive techniques with intelligent data analysis methods to facilitate the process of obtaining meaningful insights and actionable findings from large spatial-temporal datasets.However, visual analytics techniques have not solved all these problems.Even though many studies have been done, and several papers have proposed some visual exploration methods or frameworks to extract movement patterns (such as TrajGraph (Xiaoke, 2016) and TrajRank (Lu, 2015)), methods for the visual exploration of spatio-temporal relations among movement events and contexts are still absent, and the representation of the multi-scale and multi-semantic characteristics of these datasets is also not complete.The overall goal of efficiently using visual exploration methods to extract meaningful spatiotemporal relations among movement events and contexts is hindered by two challenges: a) efficiently extracting spatio-temporal relations to meet the requirements of analysts and interested parties, and b) finding an intuitive representation for spatio-temporal relations, which are multiscale and multi-semantic.
To solve the two challenges listed above, we divide our study into three steps, based on the visual analytics framework provided by (Andrienko, 2013).First, we define the specific movement events which relate to our analysis problem; for example, we may extract low speeds occurring along a movement track using an extraction method to find the stop events and stop areas.Second, we design a visualization to display multi-sematic relations within their spatio-temporal contexts.Lastly, we build an interactive multi-display interface to select the area and time interval of interest and display the movement events, as well as the spatio-temporal relations.
The major contributions of this paper are as follows:


We design a multi-view representation to visualize movement events and spatio-temporal relations, as well as the relations between movement events and the spatio-temporal context.As different scales may display different spatio-temporal relations, we design the visualization to present the relations on different levels.Note that movement trajectory points may have different semantics; for example, a GPS point may refer to a moving point using a granularity of an hour, whereas it refers to a stopping point using a granularity of a week.


We develop a prototype interactive interface to visually explore spatio-temporal relations among movement events and present two study cases that demonstrate the effectiveness and the efficiency of our approach.
The rest of the article is structured as follows: We will first review related work in Section 2. After that, we give the design considerations of our approach in Section 3, which is followed by the visual interface in Section 4. We demonstrate the effectiveness of our system with case studies in Section 5. We discuss its potentials and limitations and conclude the paper in Section 6.

Definition of movement events
General definition of movement events: an event involving the movement of one or more objects.Examples include changes in movement characteristics (speed, direction, etc.), stopping, approaching another object, and entering a particular place.Movement events are typically spatial events (Andrienko, 2013b).
The movement events consist of elements (T, S, A), and the formula is where E = movement events S= space in which the events happen T= time interval, A= events' attributes.
For example, we can define stop events as including the time and location at which the speeds of vehicles equal zero, and in which they stop for some time.
Gennady Andrienko et al. defined two classes of movement events that are extracted or derived in different way (Andrienko, 2013b): Individual movement events These events may be defined in terms of the values of movement attributes (stop events, lowspeed events, turn events, etc.).An individual movement event consists of either one point or several consecutive points of an individual's trajectory.
Collective movement events These events are defined in terms of relations between the movers: encounter events, spatial concentration events, parallel movement events, opposite movement events, etc.A collective movement event includes at least one point from the trajectory of each mover.

Definition of spatio-temporal relations:
An occurrence of a particular spatial or spatiotemporal relation among one or more movers and some elements of the spatiotemporal context, such as other movers, static spatial objects and movement events.This definition also involves how the movers are arranged in space and during instants or intervals in time of selected objects in relation to one another (Bassiri, 2008).

Movement Events Extracted
To extract the necessary movement events from movement data, a generic method is given to apply queries that set constraints on the values of positional attributes that represent the characteristics of the movement or the relations of the movers to elements of the context.In other words, the points are filtered according to the values of one or more positional attributes.
Positional attributes representing the relevant movement characteristics and/or relations can be derived beforehand from movement data or from a combination of movement data with context data.
We need to extract the characteristics of the movers, such as their speeds and sinuosities, and extract the attributes reflecting selected relations between the movers, such as the distance to the nearest neighbor and the number of neighbors within a given distance.
Domain experts desire a highly interactive and visual environment in which they can choose suitable parameters for encounter detection, such as windows that define the spatial distance ( ∆S ) and temporal distance ( ∆T ) within which an encounter occurs.Some researchers use growth ring maps (Bak, 2009) and flower diagram maps (Andrienko, 2013b) to support the investigation of cyclic temporal patterns of event occurrences in different places, which are similar to the approaches used in our work.However, the techniques above are only efficient at extracting the relations of some events that consist of the attributes of movement; the spatial and temporal semantics of movements are not always considered, especially when the requirements of the analyst changes.
There are also some interactive filtering tools introduced by Ilya Boyandin et al. (Boyandin, 2011) and that help the analyst to extract the movement events and view the results.Andrienko Natalia et al (Andrienko, 2011c) describe a visual query tool that computes and visualizes selected movement characteristics and distances.It also selects context elements and filters trajectory segments according to the values of one or more dynamic attributes.Stop events are extracted from car trajectories based on the speed values; subsequently, additional context information (e.g., point of interest) is used to separate meaningful stops from non-meaningful stops (Yan, 2009).We defined movement events using an interactive visual interface to choose the relevant attributes from the movement data.

Relations and Visualization
Growth ring maps (Bak, 2009) and flower diagram maps (Andrienko, 2013b) can also present the relations in different geometries.
To visualize the spatio-temporal relations of movers, or the relations between movement events and context, some researchers have suggested a novel interaction technique called "staining" that can be used for exploring emerging relationships between moving objects and elements of the spatio-temporal context.
There are also some visualization techniques that are commonly used in trajectory data analysis, such as space-time cubes and static maps.Tiago Gonçalves et al (Gonçalves, 2014) present a study that takes high-level visualization into account and compares these two common visualization techniques, with the result that static maps are more adequate for location tasks, while space-time cubes provide better support for association tasks.Spatio-temporal relations are strongly similar to the association tasks.Therefore, designing a space-time cube to visualize these relations may be suitable, but it is difficult to capture the information from the chaotic visual elements that may result when these relations are not actually confirmed or aggregated.
Xiaoke propose a node-link graph view to allow analysts to explore important regions.A special graph is created to store and display trajectories, which represent real traffic information, recorded by taxis driving on city streets.Our visualization is inspired by this TrajGraph, and we think that graphs represent a better visualization form to represent the relations; however, using graphs to represent spatio-temporal relations will pose some challenges when the relations contain spatial and temporal information, which makes the relations more complex.To present the spatial characteristics, graphs must connect to real positions on maps or some other geo-reference.To present the temporal characteristics, a dynamic graph is needed to represent the different moments or different time intervals involved in spatio-temporal relations.

DESIGN CONSIDERATION
In this section, we first define the spatio-temporal relations.
After that, we explain our design considerations and present the pipeline describing the steps used in applying our prototype.To analyze the spatio-temporal relations of movement events, we consider three elements:

Pipeline
To extract the spatio-temporal relations from movement events such as spatial topological relations, we first need to find the events.Many approaches exist that are based on computing the spatial and/or temporal distances between the positions of the movers and elements of the temporal context (Andrienko, 2011d).Here, we assume that we are going to extract the stop events.
We use the formula of movement events to define the taxi trajectory movement events, such as stop events and co-location events.As an example, we work to find the stop events, and we specifically consider the events in which passengers get in and get out.Every stop event takes place at a point in the space.Therefore, we need to extract the stop events from the taxi trajectory data.As is known from the trajectory data format, there is a field named 'T_Status' that contains information on whether the taxi carries passengers.Its two possible values are "1" and "0", and the first value indicates that there is no passenger in the taxi, and the second value indicates the presence of some passengers in the taxi.Thus, we look for moments when the status changes from "0" to "1", indicating that passengers exit the taxi, as well as when the status changes from "1" to "0", indicating that passengers get in the taxi.
However, the stop events' positions and distributions over different temporal intervals, and the spatial responses to these stop events, may also have different effects at different spatial scales.Therefore, we need to consider the spatial scales and temporal intervals of the relations, which are extracted from these stop events.Thus, approaches used in extracting the spatial relations should be able to re-extract and refresh the result when analysts choose smaller or larger areas.Approaches used in extracting temporal relations are the same as those used for spatial relations, and they should be able to re-extract and refresh the result when analysts change the time interval.At the same time, we should also consider periodic behaviors.In the first phase, events are extracted from the dataset.In the second phase, the three components are connected to each other, and the information is presented to the analyst.
The relations among movement events are very complex, and analysts may explore the relations at different spatial scales, which may change over large ranges.In addition, the relations may contain different means, because of the different means of movement events and nodes in the trajectory.In addition to the temporal characteristics of these relations, analysts may choose different time intervals to find different patterns.For example, if an analyst chooses one day as the time interval and the relations are grouped by week, the analyst will easily find the weekly periodicity in these relations.
These characteristics reflect the fact that the spatio-temporal relations of movement events depend on the spatial and temporal scales that are set by the analyst.In addition, when analysts obtain the information or knowledge about the relations, he/she may change the parameters to explore further.
Therefore, we think it is obvious that exploring the relations are strongly dependent on the interaction between the analyst and the computer.Except for the visualization of movement events and relations, a user-friendly interactive interface should be designed to help the analyst connect with and control the resulting relations (Gould, 2015).The main operations of the exploration process can simply be divided into two common parts, filtering and selecting.There are many techniques that are able to filter the spatial, temporal and attribute-based characteristics of relations and select the movers or groups from a graph such as a line chart or map layers, which contain features.Because movement data sets are always very large and contain more than one or two properties, it is difficult to display all the information in a static chart or graph.To solve this problem, a suitable solution involves the use of multiple displays that are connected to each other to represent different aspects of the information.
However, the communication between these displays should be flexible and efficient.More importantly, it should enable analysts to concentrate on the exploration rather than selection and filtering process.Therefore, our system needs to include a visual representation to highlight the important information from movement relations, and the pipeline of this system contains an event extraction phase and a visual exploration phase, as shown in Fig. 1.In the event extraction phase, we first define the events and set the query constraints to extract the events from the movement data.
Then, we extract the relations from the events and aggregate the relations by interactive exploration.
In terms of static and dynamic visualization, we consider that static visualizations are better than dynamic visualizations, because the study area is a city which contains detailed relations, and analysts want detail to be respected more than abstractions.Dynamic visualizations that represent long time periods and wide spatial extents may not preserve details.Thus, our design is based on static visualization, not dynamic visualization.
In the visual exploration phase, three kinds of coordinated multiple views (CMVs) are supported.CMVs represent a popular approach that has been shown to foster more creative and efficient analysis (Steed, 2013).We plot trajectories as points or lines and stop events as points or circles on a map to permit global exploration, and we use a graph to show the spatial relations among the events.To explore temporal relations, we use a time graph with time filters to select different time intervals.All these views make up a user-friendly interactive interface that allows exploration of how the information is connected to each other.
The procedure of visual exploration of the relations among movement events consists of the following steps: 1) define the events that should be extracted; 2) extract the relevant events from trajectories; 3) the analyst selects the spatial area and time intervals of interest, filters the movement events or relations to extract those that are of interest, and refreshes the overview of movement events and relations; 4) the results inspire the analyst to obtain new insights, which will affect the next exploration process.(3) Map view that shows the trajectories and events, as well as some labels with spatial information.(4) Relation Graph view, for selecting and analyzing the relations between events or objects.( 5) Time View, for aggregating and exploring the temporal distributions of events.

COORDINATED VISUAL INTERFACE
Our visual interface consists of three main views, as shown in Fig. 2: a map view for global exploration, a relation graph view, and a time view, all of which enable exploration of the spatial and temporal relations among the movement events.Visual and interactive techniques, combined with intelligent data analysis methods, can be valuable tools for obtaining useful insights and actionable findings from large spatial-temporal data sets, especially those containing movement data such as GPS trajectories.

Map View
The map view presents an intuitive overview of the city traffic network, as shown in figure 3. We follow the flow map technique and visualize the trajectory as lines.Trajectories on the map can be animated once the analyst changes the current time span.For the point events, we use flower diagram techniques, which are able to represent the position and temporal periodicity of these events.The flower diagrams also change in color or size once the analyst changes the constraints.
To highlight important information, we use a simplified map as the background of the spatio-temporal context that still contains all the necessary geo-features, such as road names and points of interest.To avoid visual clutter, the analysts can filter out the events by setting a threshold, which limits the number of events that are added to the map.

Time View
Because of the three-part structure of time (Aigner, 2007), which includes linear, cyclic, and branching time, it is difficult to visualize complex temporally distributed information and to present the means of satisfying the requirements of analysts.In this work, we only consider the linear and cyclic characteristics of the time.We use two line charts (as shown in Fig. 4) to represent the temporal information.One of these line charts represents the temporal distribution of the movement events, and another line chart under this line chart shows the interactive rectangle used to choose the range and scale of the (2) time intervals.When the x-axis of the top line chart changes, the line length will change at the same time.To represent cyclic characteristics, we use the Calendar Matrix graph (as shown in Fig. 5) to represent the periodicity of events.In this diagram, there are some units corresponding to events occurring during particular temporal intervals.For example, we can divide the day into 24 units, and each unit represents an hour of the day.The color of each unit can represent the value of some numerical attribute of the events, such as duration.
As the volume of movement events is very large and complex, we use the context and focus interaction techniques to help analysts select items of interest and focus on the process of the analysis without disturbance.

Relation Graph View
The relations among events are difficult to represent because spatial and temporal characteristics are complex; moreover, the attributes of events also make it difficult.To represent the relations between events, we use graphing techniques in which we visualize the events as nodes, the edges between nodes represent the weight between pairs of nodes as different widths, and the arrows of the edges, which represent the sequence between the two connected nodes (as shown in Fig. 6(1)), may also be different.A graph is defined as G(E,V), where V represents the set of vertices or nodes, and E defines the set of edges   with i and j ∈ V such that E⊆ V × V, in which each edge of the set E is defined by the ordered pair (  ,   ).The graph must be dynamic to represent the time-varying relations among events and their contexts.The time interval T ⊆ N , where T represents the lifetime of the defined extent of these spatio-temporal relations.
To help the analyst better understand the relations, we use force direct techniques to draw the graphs.Thus, the analyst can easily move any individual node, followed by other nodes.When it stops, the graph may will restore the layout as before.
In addition, analysts can set the force parameters to change the layout.A force can represent a classical physical force, such as electrical charges or gravity, or it can resolve a geometric constraint, such as keeping nodes within a bounding box or keeping linked nodes a fixed distance apart.Another visualization diagram is a matrix-graph, in which relations are represented as the units of a matrix.Rows and columns represent the same set of movers or groups.The shades of the units' colors stand for the strength of the relation between the movers in a row and column.

CASE STUDIES
In this section, we implement a prototype of our visual exploration, and we design two study cases which use taxi trajectory data from Wuhan to demonstrate the effectiveness and efficiency of our approach.The first case explores the relations among stop events, and the second case explores the relations between events and their contexts.

Data Description
In the past few years, the government of China has been promoting a project named Smarter City that involves the transportation system of the city of Wuhan.In this project, several thousand cars and buses have been equipped with GPS and connected to the network with others.Intelligent transportation construction is the foundation of intelligent traffic.The wisdom of perception, intelligence analysis, and intelligent decision-making are keys to intelligent transportation.The development of intelligence analysis related to transportation decides the progress of transportation, and it will make transportation more efficient, more convenient, and more helpful to the citizens.
Table 1.Main data types and sources Our dataset contains two parts, which contain trajectory data and road data.The trajectory data are derived from the taxi GPS units and tagged with information from the taxi drivers.We obtained about one year of taxi GPS trajectory data, which were produced by approximately 4000 cars in 2013.The format of the trajectory data is <T_TID, T_UTCTime, T_Lon, T_Lat, T_Speed, T_Heading, T_Status> Each row within this data set represents a point in the taxi's trajectory, the ID in each row identifies the taxi, and each point is tagged with a time stamp that denotes the point in time corresponding to the car's position.The speed field denotes the car's speed at that moment, and status reflects whether or not the car is carrying passengers.
The format of the information on each taxi is <T_TID, T_TNumber, T_Name, T_Telephone> Each row of these data presents basic information on one taxi.The ID field is mapped to the ID of the trajectory data, number denotes the taxi number, and name denotes the name of the driver.Telephone denotes the telephone number of the driver.For the road data, we used the road data which we downloaded from OpenStreetMap.The scope of this road data covers central Wuhan.Its format is <ID, SHAPE, highway, name > Each row of these data presents a geometry object corresponding to a road.The ID field is the identifier of the object, the shape field presents the type of the object, highway denotes the level of the road, and name denotes the road's name.Moreover, we also need some tile maps for use as the background of these trajectories to show the spatial relations between movers and their spatio-temporal context.Our trajectory data spans approximately 83 days from Jan. 10 th 2013 to Aug 25 th 2013.Taken together, there are 240 million records on over 4000 taxis.The data size is 20 GB.

Visual exploration of the relations between stop events
The stop events contain information on taxi use, which can represent the traffic distribution of the city.They also contain information on when and where passengers get into or out of the taxis.
To explore the relations between these stop events, such as spatial relations, temporal relations, etc., we divide the relations among these events into two categories:  Relations among the stop events derived from a single trajectory.


Relations among stop events which are derived from different movers or groups of taxi trajectory data.
We choose taxi data collected on 28 th May 2013 and extract the stop events, during which the speed of the taxi equals zero.
As we want to analyze the relations among stop events on Luo Yu Street, we choose a single trajectory and select the stop points on the map.We choose the taxis whose TIDs are 20196, 18308, and 19701, and their temporal distributions are shown in a time view (Fig. 7).
From the time graph, we can see that all three of these taxis have similar distributions of events.Most of the stop events are concentrated from 3PM to 9PM.Thus, the time relations between these trajectories are easy to understand.Combined with the map view, the trajectory of taxi_20196 is mainly on Guan Shan Street, taxi_18308 is mainly on Han Kou District, and taxi_19701 is mainly on Wu Chang District.
Figure 8.The panel shows an overview of the stop events, and the right-hand panel In the case, we want to determine the relations between taxi_18308 and taxi_20196, such as where they "meet" on Luo Yu Street.The taxi_18308 passes along Luo Yu Street at 8:25~8:42, and taxi_20196 passes along this street at almost the same position with taxi_18308.The spatial and temporal relations between these stop events can be seen in Fig. 9.
Figure 9. (1) Relation View: The yellow, green and blue circles represent stop events related to both taxis.Ticks on the edges between nodes reflect the strong dependence between these events.(2) Map View: Yellow circles represent the stop events related to taxi_18308, and blue circles presents the stop events of taxi_20196.The taxis both enter the rectangle but not at the same moment.
They both stop at Wuhan railway station and near the hospital of Guangzhou Military Region.

Visual exploration of the relations between events and their spatio-temporal context
As for the relations between events and their contexts, we want to determine the event relations between the street and the stop events.In this case, we choose Luo Yu Street as the context and use the flow diagram to determine the relations between this street and the stop events.
We divide the flower into 24 sectors, and each sector represents an hour of the day.The length of the sector represents the number of stop events.As shown in figure 10, most stop events occur during the morning and the evening, because the Luo Yu Street is always congested at those times.Near road crossings, many taxis and cars are waiting for the traffic lights.Moreover, we also think that the stop events not only relate to the streets but also to the crossroads and trunk roads.If we obtain the spatial distribution of traffic lights, we can explore the relations between these events and traffic lights, as well as some other transportation facilities.
Figure 10.Relations between events and the street.The street is shown in red.

DISCUSSION AND CONCLUSION
In this paper, we have provided a novel approach to visually explore the relations, especially spatio-temporal relations, among movement events and their spatio-temporal contexts.We design an interactive visual tool to extract the events.To explore the information at different spatial and temporal scales, we design three fundamental multi-display interfaces that are connected and synchronized to visualize the findings in different dimensions, and use interactions involving context and focus to look for insights.
To demonstrate the effectiveness and efficiency of this approach, we implement a prototype tool for visual exploration and design two case studies using taxi trajectory data from Wuhan city.The results show that our approach is well suited to explore the (2) (1) Near the hospital of Guangzhou Military Region Wuhan railway station relations among movement events.We think this prototype will extend to distributed computation environments, and we hope it will inspire readers and researchers.

Figure 1 .
Figure1.The pipeline of the prototype method contains two phases: the extraction of movement events and interactive visual exploration.In the first phase, events are extracted from the dataset.In the second phase, the three components are connected to each other, and the information is presented to the analyst.

Figure 2 .
Figure 2. Interface of the prototype: (1) Side toolbar for navigation.(2) Parameter controller.(3)Map view that shows the trajectories and events, as well as some labels with spatial information.(4) Relation Graph view, for selecting and analyzing the relations between events or objects.(5) Time View, for aggregating and exploring the temporal distributions of events.4.COORDINATED VISUAL INTERFACE

Figure 3 .
Figure 3. Movement events are represented by circles with different colors.

Figure 4
Figure 4 Time View for displaying temporal information, (1) time-varying values.(2) distributions of movement events.

Figure 5 .
Figure 5. Calendar Matrix graph for displaying periodicity