Ismail Ari

Managing flash crowds on the Internet

A flash crowd is a surge in traffic to a particular Web site that causes the site to be virtually unreachable. We present a model of flash crowd events and evaluate the performance of various multilevel caching techniques suitable for managing these events. By using well-dispersed caches and with judicious choice of replacement algorithms we show reductions in client response times by as much as a factor of 25. We also show that these caches eliminate the server and network hot spots by distributing the load over the entire network.

A Survey of Software Testing in the Cloud

Cloud computing has emerged as a new computing paradigm that impacts several different research fields, including software testing. Testing cloud applications has its own peculiarities that demand for novel testing methods and tools. On the other hand, cloud computing also facilitates and provides opportunities for the development of more effective and scalable software testing techniques. This paper reports on a systematic survey of published results attained by the synergy of these two research fields. We provide an overview regarding main contributions, trends, gaps, opportunities, challenges and possible research directions. We provide a review of software testing over the cloud literature and categorize the body of work in the field.

E-Cube: multi-dimensional event sequence analysis using hierarchical pattern query sharing

Many modern applications, including online financial feeds, tag-based mass transit systems and RFID-based supply chain management systems transmit real-time data streams. There is a need for event stream processing technology to analyze this vast amount of sequential data to enable online operational decision making. Existing techniques such as traditional online analytical processing (OLAP) systems are not designed for real-time pattern-based operations, while state-of-the-art Complex Event Processing (CEP) systems designed for sequence detection do not support OLAP operations. We propose a novel E-Cube model which combines CEP and OLAP techniques for efficient multi-dimensional event pattern analysis at different abstraction levels. Our analysis of the interrelationships in both concept abstraction and pattern refinement among queries facilitates the composition of these queries into an integrated E-Cube hierarchy. Based on this E-Cube hierarchy, strategies of drill-down (refinement from abstract to more specific patterns) and of roll-up (generalization from specific to more abstract patterns) are developed for the efficient workload evaluation. Our proposed execution strategies reuse intermediate results along both the concept and the pattern refinement relationships between queries. Based on this foundation, we design a cost-driven adaptive optimizer called Chase, that exploits the above reuse strategies for optimal E-Cube hierarchy execution. Our experimental studies comparing alternate strategies on a real world financial data stream under different workload conditions demonstrate the superiority of the Chase method. In particular, our Chase execution in many cases performs ten fold faster than the state-of-the art strategy for real stock market query workloads.

High-performance nested CEP query processing over event streams

Complex event processing (CEP) over event streams has become increasingly important for real-time applications ranging from health care, supply chain management to business intelligence. These monitoring applications submit complex queries to track sequences of events that match a given pattern. As these systems mature the need for increasingly complex nested sequence query support arises, while the state-of-art CEP systems mostly support the execution of flat sequence queries only. To assure real-time responsiveness and scalability for pattern detection even on huge volume high-speed streams, efficient processing techniques must be designed. In this paper, we first analyze the prevailing nested pattern query processing strategy and identify several serious shortcomings. Not only are substantial subsequences first constructed just to be subsequently discarded, but also opportunities for shared execution of nested subexpressions are overlooked. As foundation, we introduce NEEL, a CEP query language for expressing nested CEP pattern queries composed of sequence, negation, AND and OR operators. To overcome deficiencies, we design rewriting rules for pushing negation into inner subexpressions. Next, we devise a normalization procedure that employs these rules for flattening a nested complex event expression. To conserve CPU and memory consumption, we propose several strategies for efficient shared processing of groups of normalized NEEL subexpressions. These strategies include prefix caching, suffix clustering and customized “bit-marking” execution strategies. We design an optimizer to partition the set of all CEP subexpressions in a NEEL normal form into groups, each of which can then be mapped to one of our shared execution operators. Lastly, we evaluate our technologies by conducting a performance study to assess the CPU processing time using real-world stock trades data. Our results confirm that our NEEL execution in many cases performs 100 fold faster than the traditional iterative nested execution strategy for real stock market query workloads.

CHAOS: A Data Stream Analysis Architecture for Enterprise Applications

In this paper, we describe the design of our architecture for Continuous, Heterogeneous Analysis Over Streams, aka CHAOS that combines stream processing, approximation techniques, mining, complex event processing and visualization. CHAOS, with the novel concept of Computational Stream Analysis Cube, provides an effective, scalable platform for near real time processing of business and enterprise streams. We describe our approach with a real data center temperature analysis application.

E-Cube: Multi-dimensional event sequence processing using concept and pattern hierarchies

Many modern applications including tag based mass transit systems, RFID-based supply chain management systems and online financial feeds require special purpose event stream processing technology to analyze vast amounts of sequential multi-dimensional data available in real-time data feeds. Traditional online analytical processing (OLAP) systems are not designed for real-time pattern-based operations, while Complex Event Processing (CEP) systems are designed for sequence detection and do not support OLAP operations. We will demonstrate a novel E-Cube model that combines CEP and OLAP techniques for multi-dimensional event pattern analysis at different abstraction levels. A London transit scenario will be given to demonstrate the utility and performance of this proposed technology.

SANBoost: automated SAN-level caching in storage area network

The storage traffic for different logical units (LUs) of a disk array converge at the arrays cache. The cache is allocated among the LUs approximately according to their relative I/O rates. In the case of nonuniform I/O rates and sensitivity to storage response times between differing applications in a storage area network (SAN), undesirable cache interference between LUs can result in unacceptable storage performance for some LUs. This paper describes SANBoost, a SAN-level caching approach that can be enabled selectively on a per-LU basis to provide a performance isolation mechanism for response time metrics related to storage quality of service (QoS). SANBoost automates hot data detection and migration processes in block-level storage. The design consists of a migration module implemented in a fabric-based SAN virtualization appliance and a solid-state disk (SSD) that acts as a cache resource within the same SAN. Simulation results quantify the impact of a specific static SANBoost caching policy on the SPC-1 benchmark workload and address the relative impact of adapting a threshold in the placement algorithm.

Data stream analytics and mining in the cloud

Due to prevalent use of sensors and network monitoring tools, big volumes of data or “big data” today traverse the enterprise data processing pipelines in a streaming fashion. While some companies prefer to deploy their data processing infrastructures and services as private clouds, others completely outsource these services to public clouds. In either case, attempting to store the data first for subsequent analysis creates additional resource costs and unwanted delays in obtaining actionable information. As a result, enterprises increasingly employ data or event stream processing systems and further want to extend them with complex online analytic and mining capabilities. In this paper, we present implementation details for doing both correlation analysis and association rule mining (ARM) over streams. Specifically, we implement Pearson-Product Moment Correlation for analytics and Apriori & FPGrowth algorithms for stream mining inside a popular event stream processing engine called Esper. As a unique contribution, we conduct experiments and present performance results of these new tools with different tumbling and sliding time-windows over two different stream types: one for moving bus trajectories and another for web logs from a music site. We find that while tumbling windows may be more preferable for performance in certain applications, sliding windows can provide additional benefits with rule mining. We hope that our findings can shed light on the design of other cloud analytics systems.

Processing nested complex sequence pattern queries over event streams

Complex event processing (CEP) has become increasingly important for tracking and monitoring applications ranging from health care, supply chain management to surveillance. These monitoring applications submit complex event queries to track sequences of events that match a given pattern. As these systems mature the need for increasingly complex nested sequence queries arises, while the state-of-the-art CEP systems mostly focus on the execution of flat sequence queries only. In this paper, we now introduce an iterative execution strategy for nested CEP queries composed of sequence, negation, AND and OR operators. Lastly we have introduced the promising direction of applying selective caching of intermediate results to optimize the execution. Our experimental study using real-world stock trades evaluates the performance of our proposed iterative execution strategy for different query types.

Mobile In-store Personalized Services

The Mobile Shopping Assistant (MSA) is a mobile application platform to deliver real-time, in-store, and personalized services, such as personalized product offerings and in-store customer advisory support, to improve the shopping experiences of in-store customers. The service delivery network that powers the MSA involves retail stores and their business partners such as manufacturers. This paper presents the core technologies that we developed in this cross-organizational service network to support the MSA and its personalized services, with focus on service delivery, customer behavior understanding and information sharing. Our event-based techniques allow customers, stores and manufacturers to deliver and consume the services in a loosely coupled manner, thus solving a critical store-specific real-time engagement problem in a seamless way. Service response tracking enables the stores to construct a comprehensive view of a customer’s in-store shopping behavior. Finally, the cross-organizational authorization-based access control mechanism effectively enforces information sharing between the stores and their partners.