Zdravko Galić

Geospatial data streams: Formal framework and implementation

Abstract A spatio-temporal database manages spatio-temporal objects and supports corresponding query languages. Today, the term moving objects databases is used as a synonym for spatio-temporal databases managing spatial objects with a continuously changing geospatial location and/or extent. Recent advances in wireless communication, miniaturization of spatially enabled devices and global navigation satellite systems (GNSS) services have resulted in a large number of novel application domains. Applications in these novel domains (geo-sensor networks, moving objects tracking, real-time traffic analysis, etc.) process huge volumes of continuous data streams, i.e. data sets that are produced incrementally over time, rather than those available in full before the processing begins. Several data stream management systems (DSMSs) have been developed to manage this data. Since they are mainly based on a relational paradigm, they do not support geospatial data. Therefore, there is an urgent need for geospatial data stream management, ranging from real-time monitoring and alerting to long-term analysis of processed geospatial data. In this paper we present a formal framework consisting of data types and operations needed to support geospatial data in data streams. It can be used as a basis either for implementation of a completely new geospatial DSMS, or for extending available open source products and research prototypes. We leverage the work on abstract data types from spatio-temporal databases, present an implementation based on user-defined aggregate functions and illustrate embedding into an SQL-like language.

OCEANUS: a spatio-temporal data stream system prototype

Recent advances in wireless communication, miniaturization of spatially enabled devices and global navigation satellite systems (GNSS) services have resulted in a large number of novel application domains. Applications in these novel domains (moving objects tracking, sensor networks, fleet management, real-time intelligent transportation systems, etc.) process huge volume of continuous streaming data, i.e. data that is produced incrementally over time, rather than being available in full before processing. Data stream management systems (DSMS) have been developed to manage continuous data streams. Usually based on relational paradigm, they have rudimentary support for spatial data. Recent research efforts in data stream management systems focus mainly on processing continuous queries over traditional data streams, and only a few papers addressed spatio-temporal continuous queries. In this paper we present OCEANUS, an ongoing effort to extend TelegraphCQ DSMS with spatial support providing a platform for spatio-temporal streaming applications.

Distributed processing of big mobility data as spatio-temporal data streams

Recent rapid development of wireless communication, mobile computing, global navigation satellite systems (GNSS), and spatially enabled sensors are leading to an exponential growth of available mobility data produced continuously at high speed. Due to these advancements, a new class of monitoring applications has come to the focus, including real-time intelligent transportation systems, traffic monitoring and mobile objects tracking. These new information flow processing (IFP) application domains need to process huge volume of mobility data arriving in the form of continuous data streams from mobile objects. IFP applications are pushing traditional database technologies beyond their limits due to their massively increasing data volumes and demands for real-time processing. Mobility data, i.e. real-time, transient, time-varying sequences of spatio-temporal data items, generated by embedded positioning sensors demonstrates at least two Big Data core features: volume and velocity. Existing distributed data stream management systems (DSMS), real-time computing systems (RTCS) and their processing models are dominantly based on relational paradigm and continuous operator model. Thus, they have rudimentary spatio-temporal capabilities, provide expensive fault recovery requiring either hot replication or long recovery times, and do not handle faults and slow nodes. The framework proposed in this paper is a cornerstone towards efficient real-time managing and monitoring of mobile objects through distributed spatio-temporal streams processing on large clusters. A prototype implementation is rooted in a new stream processing model that overcomes the challenges of current distributed stream processing models and enable seamless integration with batch and interactive processing like MapReduce.

Data Types and Operations for Spatio-temporal Data Streams

While the field of spatio-temporal databases, moving objects databases and the field of data stream management systems have separately been rather extensively researched in the last decade, a combined field of spatio-temporal data stream management systems has only recently become a focus of serious research effort. A spatio-temporal data stream management system manages moving objects whose position and/or extent is provided by one or more data streams, and can be viewed as an extension of a moving objects database enabling it to manage data streams, or as an extension of a data stream management system enabling it to use spatial and spatio-temporal data. In this paper we present a data model, consisting of data types and operations on these data types, needed to support spatio-temporal data in a data stream management system. Presented data model is formally defined using many sorted algebra, and is illustrated through an SQL-like query language. It can be used as a basis either for implementation completely new spatio-temporal DSMS, or for extending available open source products and research prototypes, towards managing moving objects.

Spatio-Temporal Continuous Queries

Spatio-temporal stream processing in general refers to a class of software systems for processing of high volume spatio-temporal data streams with very low latency, i.e. in near real-time. Motivated by the limitation of DBMS , the database community developed data stream management systems (DSMSs), as a new class of management systems oriented toward processing large data streams in a near real-time. Despite differences these between these two classes of management systems, DSMSs resemble DBMSs—they process data streams using SQL and operators defined by the relational algebra. This chapter gives an insight into spatio-temporal stream processing at conceptual level, i.e. from the DSMS user perspective.

Spatio-Temporal Data Stream Clustering

Spatio-temporal data streams are huge amounts of data pertaining to time and position of moving objects. Mining such amount of data is a challenging problem, since the possibility to extract useful information from this peculiar kind of data is crucial in many RFIP application scenarios. Moreover, spatio-temporal data streams pose interesting challenges for their proper representation, thus making the mining process harder than for classical data. In this chapter we deal with a specific spatio-temporal data stream class, namely trajectory streams that collect data pertaining to spatial and temporal position of mobile objects.

Spatio-Temporal Data Streams and Big Data Paradigm

Recent rapid development of wireless communication, mobile computing, global navigational satellite systems (GNSS), and spatially enabled sensors is leading to an exponential growth of available spatio-temporal data produced continuously at hight speed. Spatio-temporal data streams, i.e. real-time, transient, time-varying sequences of spatiotemporal data items, demonstrates at least two Big Data core features: volume and velocity. To handle the volumes of data and computation they involve, these applications need to be distributed over clusters. However, despite substantial work on cluster programming models for batch computation, there are few similarly high-level tools for stream processing. Obviously, there is a clear need for highly scalable spatio-temporal stream computing framework that can operate at high data rates and process massive amounts of big spatio-temporal data streams. In this chapter we present our approach and framework for an integrated big spatio-temporal data stream processing. The key concept here is that streaming data and persistent data are not intrinsically different - the persistent spatio-temporal data is simply streaming data that has been entered into the persistent structures.

Generating spatio-temporal streaming trajectories

Recent research efforts in data stream management systems (DSMS) focus mainly on processing continuous queries over traditional data streams, and only a few addressed spatio-temporal continuous queries. OCEANUS presents an effort to extend TelegraphCQ DSMS with spatial support providing a platform for spatio-temporal streaming applications. Data type system that represents the formal basis for modeling moving objects in data streams, as well as an approach for managing moving objects in spatio-temporal data streams based on user-defined aggregate functions (UDAF) are presented. In this paper we are concerned with improving proposed approach for extracting moving objects out of spatio-temporal data streams based on UDAF. It is based on two methods for detecting static locations on trajectories of moving objects and for reducing total number of units in sliced representation by introducing trajectory buffer. First method leads to more precise results of certain operations over moving objects such as retrieving information about speed and direction, but also enables introduction of few new operations such as retrieving static locations and overall stoppage time on moving objects trajectory. Second method leads to better memory usage during processing spatio-temporal continuous queries in DSMS.