Qingchun Jiang

Scheduling Strategies for Processing Continuous Queries over Streams

Stream data processing poses many challenges. Two important characteristics of stream data processing – bursty arrival rates and the need for near real-time performance requirement – challenge the allocation of limited resources in the system. Several scheduling algorithms (e.g., Chain strategy) have been proposed for minimizing the maximal memory requirements in the literature. In this paper, we propose novel scheduling strategies to minimize tuple latency as well as total memory requirement. We first introduce a path capacity strategy (PCS) with the goal of minimizing tuple latency. We then compare the PCS and the Chain strategy to identify their limitations and propose additional scheduling strategies that improve upon them. Specifically, we introduce a segment strategy (SS) with the goal of minimizing the memory requirement, and its simplified version. In addition, we introduce a hybrid strategy, termed the threshold strategy (TS), to addresses the combined optimization of both tuple latency and memory requirement. Finally, we present the results of a wide range of experiments conducted to evaluate the efficiency and the effectiveness of the proposed scheduling strategies.

Data stream management system for MavHome

MavHome project provides rich applications for addressing various issues associated with stream data processing. In this paper, we present our approach for building a data stream management system (DSMS) for the above smart home project. We further summarize our primitive solutions for continuous query processing, Quality of Service(QoS) management, and mapping a trigger mechanism to a stream processing system.

Queueing analysis of relational operators for continuous data streams

Currently, stream data processing is an active area of research, which includes everything from algorithms and architectures for stream processing to modelling, and analysis of various components of a stream processing system. In this paper, we present an analysis of relational operators used for stream processing using queueing theory and study behaviors of streaming data in a query processing system. Our approach enables us to compute the fundamental performance metrics of relational operators ---select, project, and join over data streams. Furthermore, this approach establishes a way to find the probability distribution functions of both the number of tuples and the waiting time of tuples in the system. Finally, we designed and implemented a number of experiments to validate the accuracy and effectiveness of our analysis.

MavEStream: Synergistic Integration of Stream and Event Processing

Although research seems to address event and stream data processing as two separate topics, there are a number of similarities between them. For many advanced stream applications, both event and rule processing are needed and are not currently well-supported. Extant event processing systems concentrate primarily on complex events and rules and stream processing systems concentrate on stream operators, scheduling, and quality of service issues. Synergistic integration of these models will be better than the sum of its parts. We propose an integrated model to combine the capabilities of both models for applications that need both of them. Specifically, we introduced a number of enhancements, including stream modifiers, semantic windows, event generators, and enhanced event and rule specifications, to couple two models seamlessly. We prototype our integrated system using the stream processing system (MavStream) with the event processing system (Snoop and Sentinel) and discuss the design and implementation issues of our prototype.