Anton V. Riabov

IBM infosphere streams for scalable, real-time, intelligent transportation services

With the widespread adoption of location tracking technologies like GPS, the domain of intelligent transportation services has seen growing interest in the last few years. Services in this domain make use of real-time location-based data from a variety of sources, combine this data with static location-based data such as maps and points of interest databases, and provide useful information to end-users. Some of the major challenges in this domain include i) scalability, in terms of processing large volumes of real-time and static data; ii) extensibility, in terms of being able to add new kinds of analyses on the data rapidly, and iii) user interaction, in terms of being able to support different kinds of one-time and continuous queries from the end-user. In this paper, we demonstrate the use of IBM InfoSphere Streams, a scalable stream processing platform, for tackling these challenges. We describe a prototype system that generates dynamic, multi-faceted views of transportation information for the city of Stockholm, using real vehicle GPS and road-network data. The system also continuously derives current traffic statistics, and provides useful value-added information such as shortest-time routes from real-time observed and inferred traffic conditions. Our performance experiments illustrate the scalability of the system. For instance, our system can process over 120000 incoming GPS points per second, combine it with a map containing over 600,000 links, continuously generate different kinds of traffic statistics and answer user queries.

A Folksonomy-Based Model of Web Services for Discovery and Automatic Composition

In this paper, we propose a novel way of modeling Web services using folksonomies. The key advantage of our model is that it allows a large number of users to participate, easily, in annotating services with tags. This is in contrast to more expressive, logic based models of services, such as semantic Web service models, which require significant expertise for annotation and maintenance. Our folksonomy-based model allows associating semantic constraints on the input and output messages of web service operations using tags obtained from a folksonomy. We show how the model can be used for discovery and composition of services. We also describe a planner that uses this model to compose services and create workflows, automatically. We present performance results for the planner and our experiences in using this model in a sample real-world domain.

A semantics-based middleware for utilizing heterogeneous sensor networks

With the proliferation of various kinds of sensor networks, we will see large amounts of heterogeneous data. They have different characteristics such as data content, formats, modality and quality. Existing research has largely focused on issues related to individual sensor networks; how to make use of diverse data beyond the individual network level is largely unaddressed. In this paper, we propose a semantics-based approach for this problem and describe a system that constructs applications that utilize many sources of data simultaneously. We propose models to formally describe the semantics of data sources, and processing modules that perform various kinds of operations on data. Based on such formal semantics, our system composes data sources and processing modules together in response to users queries. The semantics provides a common ground such that data sources and processing modules from various parties can be shared and reused among applications. We describe our system architecture, illustrate application deployment, and share our experiences in the semantic approach.

Scalability of reliable group communication using overlays

This study provides some new insights into the scalability of reliable group communication mechanisms using overlays. These mechanisms use individual TCP connections for packet transfers between end-systems. End-systems store incoming packets and forward them to downstream nodes using different unicast TCP connections. In this paper we assume that buffers in end-systems are large enough for the transfers. It is shown that the throughput of the reliable overlay group communication scales in the sense that for all multicast tree sizes and topologies, the group throughput is strictly positive under natural conditions. This is in contrast with the IP supported multicast paradigm where reliable protocols have vanishing throughput when the group size tends to infinity. The scalability of packet delay and buffer occupancy is then investigated. In the absence of additional control, the occupancy of the buffer and the latency in the end-systems explodes with time. It is then shown that proactive rate throttle mechanism implemented at the source leads to finite packet latency and buffer occupancy in any end-system of the network provided certain moment conditions are satisfied by cross traffic in the routers.

ModelingWeb Services using Semantic Graph Transformations to aid Automatic Composition

In this paper, we propose a novel way of modeling Web services using semantic graph transformations. Each operation supported by a Web service is associated with a semantic annotation that describes the input and output messages using RDF graph patterns. The terms used in these patterns are defined in OWL ontologies that describe the application domain. A key difference between our model and existing semantic Web service models like OWLS is that it describes the inputs and outputs in terms of instance-based graph patterns, rather than in terms of concepts. This allows associating a rich set of constraints on the input and output data in terms of relations between instances. We also propose a composition model for Web service operations, that describes the conditions for composing services into workflows. The composition model includes the notion of semantic propagation, i.e. the semantic description of the output message of an operation depends on the semantics of the input message. We have developed a planner that uses this model to compose services, automatically. The planner uses DLP reasoning to aid plan search. We present performance results for the planner.

A tag-based approach for the design and composition of information processing applications

In the realm of component-based software systems, pursuers of the holy grail of automated application composition face many significant challenges. In this paper we argue that, while the general problem of automated composition in response to high-level goal statements is indeed very difficult to solve, we can realize composition in a restricted context, supporting varying degrees of manual to automated assembly for specific types of applications. We propose a novel paradigm for composition in flow-based information processing systems, where application design and component development are facilitated by the pervasive use of faceted, tag-based descriptions of processing goals, of component capabilities, and of structural patterns of families of application. The facets and tags represent different dimensions of both data and processing, where each facet is modeled as a finite set of tags that are defined in a controlled folksonomy. All data flowing through the system, as well as the functional capabilities of components are described using tags. A customized AI planner is used to automatically build an application, in the form of a flow of components, given a high-level goal specification in the form of a set of tags. End-users use an automatically populated faceted search and navigation mechanism to construct these high-level goals. We also propose a novel software engineering methodology to design and develop a set of reusable, well-described components that can be assembled into a variety of applications. With examples from a case study in the Financial Services domain, we demonstrate that composition using a faceted, tag-based application design is not only possible, but also extremely useful in helping end-users create situational applications from a wide variety of available components.

The one-to-many TCP overlay: a scalable and reliable multicast architecture

We consider reliable multicast in overlay networks where nodes have finite-size buffers and are subject to failures. We address issues of end-to-end reliability and throughput scalability in this framework. We propose a simple architecture which consists of using distinct point-to-point TCP connections between adjacent pairs of end-systems, together with a back-pressure control mechanism regulating the transfers of adjacent TCP connections, as well as a back-up buffering system handling node failures. This architecture, that we call the one-to-many TCP overlay, is a natural extension of TCP to the one-to-many case, in that it adapts the rate of the group communication to local congestion in a decentralized way via the window back-pressure mechanism. Using theoretical investigations, experimentations in the Internet, and large network simulations, we show that this architecture provides end-to-end reliability and can tolerate multiple simultaneous node failures, provided the backup buffers are sized appropriately. We also show that under random perturbations caused by cross traffic described in the paper, the throughput of this reliable group communication is always larger than a positive constant, that does not depend on the group size. This scalability result contrasts with known results about the non-scalability of IP-supported multicast for reliable group communication.

Mashup-based information retrieval for domain experts

In this paper, we tackle the problem of helping domain experts to construct, parameterize and deploy mashups of data and code. We view a mashup as a data processing flow, that describes how data is obtained from one or more sources, processed by one or more components, and finally sent to one or more sinks. Our approach allows specifying patterns of flows, in a language called Cascade. The patterns cover different possible variations of the flows, including variations in the structure of the flow, the components in the flow and the possible parameterizations of these components. We present a tool that makes use of this knowledge of flow patterns and associated metadata to allow domain experts to explore the space of possible flows described in the pattern. The tool uses an AI planning approach to automatically build a flow, belonging to the flow pattern, from a high-level goal, specified as a set of tags. We describe examples from the financial services domain to show the use of flow patterns in allowing domain experts to construct a large variety of mashups rapidly.

Specifying and Enforcing High-Level Semantic Obligation Policies

Obligation policies specify management actions that must be performed when a particular kind of event occurs and certain conditions are satisfied. Large scale distributed systems often produce event streams containing large volumes of low-level events. In many cases, these streams also contain multimedia data (consisting of text, audio or video). Hence, a key challenge is to allow policy writers to specify obligation policies based on high-level events, that may be derived after performing appropriate processing on raw, low-level events. In this paper, we propose a semantic obligation policy specification language called Eagle, which is based on patterns of high-level events, represented as RDF graph patterns. Our policy enforcement architecture uses a compiler that builds a workflow for producing a stream of events, which match the high-level event pattern specified in a policy. This workflow consists of a number of event sources and event processing components, which are described semantically. We present the policy language and enforcement architecture in this paper.

Stream Processing Based Intelligent Transport Systems

In this paper, we present a Fleet Management Center application implemented using a stream processing infrastructure we call System S. System S enables the deployment of large scale applications with mechanisms for sharing multi-party data sources, software components, and even intermediate results. This approach significantly reduces the cost of software integration, and ownership, the major factor in Intelligent Transportation Systems. In addition, the system includes an adaptive data source management that determines the list of relevant data sources based on the current locations of the entities monitored or managed by the applications.