Daniel C. Sturman

Matching events in a content-based subscription system

Content-based subscription systems are an emerging alternative to traditional publish-subscribe systems, because they permit more flexible subscriptions along multiple dimensions. In these systems, each subscription is a predicate which may test arbitrary attributes within an event. However, the matching problem for content-based systems — determining for each event the subset of all subscriptions whose predicates match the event — is still an open problem. We present an efficient, scalable solution to the matching problem. Our solution has an expected time complexity that is sub-linear in the number of subscriptions, and it has a space complexity that is linear. Specifically, we prove that for predicates reducible to conjunctions of elementary tests, the expected time to match a random event is no greater than O(N 1 ) where N is the number of subscriptions, and is a closed-form expression that depends on the number and type of attributes (in some cases, 1=2). We present some optimizations to our algorithms that improve the search time. We also present the results of simulations that validate the theoretical bounds and that show acceptable performance levels for tens of thousands of subscriptions. Department of Computer Science, Cornell University, Ithaca, N.Y. 14853-7501, aguilera@cs.cornell.edu IBM T.J. Watson Research Center, Yorktown Heights, N.Y. 10598, fstrom, sturman, tusharg@watson.ibm.com Department of Computer Science, University of Illinois at Urbana-Champaign, 1304 W. Springfield Ave, Urbana, I.L. 61801, astley@cs.uiuc.edu

Exploiting IP multicast in content-based publish-subscribe systems

Publish-subscribe systems are evolving toward using content-based subscription rather than subject-based subscription. A key problem in implementing such systems is that a straightforward mapping from matching sets to multicast groups produces a number of groups that rapidly grows beyond practical limits. This paper proposes a set of alternative algorithms for solving this problem, by: (1) using a smaller set of overbroad multicast groups, judiciously chosen to minimize imprecision; (2) issuing multiple multicasts to appropriately chosen clusters; or (3) sending an event over multiple hops each involving a multicast to a set of neighbors. We evaluate these algorithms on a simulated wide-area network. We find that (1) a simple flooding algorithm is viable over an extensive range of conditions; and (2) under conditions of high selectivity and high regionalism of subscriptions, the other approaches mentioned above perform significantly better; however, the specific algorithm to use depends upon the economics of deployment.

A Case for Message Oriented Middleware

With the emergence of the internet, independent applications are starting to be integrated with each other. This has created a need for technology for glueing together applications both within and across organizations, without having to re-engineer individual components. We propose an approach for developing this glue technology based on message flows and discuss the open research problems in realizing this approach.

A gossip-based reliable multicast for large-scale high-throughput applications

Group-based reliable multicast is an important building block for distributed applications. For large systems, however, traditional approaches do not scale well due to centralized recovery mechanisms and excessive message overhead. In this paper, we present a reliable probabilistic multicast, rpbcast, that is a hybrid of the centralized and gossip-based approaches. In particular, rpbcast extends previous work by supporting high packet rates and many active senders. Rpbcast uses gossip as the primary retransmission mechanism and only contacts loggers if gossips fail. Large groups of active senders are supported using negative gossip that specifies those messages a receiver is missing instead of those messages it received. Moreover, we show that negative gossip allows pull based recovery and converges faster than push based recovery. Rpbcast also applies hashing techniques to reduce message overhead and approximate group membership for garbage collection. We describe the key features of rpbcast and present simulation results.

Dynamic Access Control in a Content-based Publish/Subscribe System with Delivery Guarantees

Content-based publish/subscribe (pub/sub) is a promising paradigm for building asynchronous distributed applications. In many application scenarios, these systems are required to provide stringent service guarantees such as reliable delivery, high performance, high availability and dynamic system security. In this paper, we address the issue of dynamic access control in a content-based system that provides reliable delivery and high availability through redundant routes. We define a deterministic service model of dynamic access controls that enables precise control over event confidentiality. Under this model, the semantics of reliable delivery is clearly defined, that is, the messages delivered in response to the same subscriptions from pub/sub clients running on behalf of the same principal will be exactly the same, regardless of their connecting locations, network latency and failures. We present an algorithm that implements this service model. The algorithm is efficient and highly available in that it enables uniform enforcement of access control and enables content-based routing to choose any path from among several redundant routes without requiring consensus among the brokers.

Towards continuous availability of Internet services through availability domains

The increasing number of Internet users has caused a dramatic increase in electronic commerce. This growth is outpacing technologies for dependability, causing traditional views of high availability to come under question. In particular, Internet failures are a phenomenon external to the owner of a commerce site that must be dealt with, and therefore, geographically distributed servers are a basic availability requirement for e-commerce sites. Geographic distribution provides an opportunity to view users in different roles based on those distributed components they must access. This paper presents an approach based on partitioning online function into domains, each of which provides service to users in a specific role. Coordination between domains is eliminated as much as possible by exploiting application-specific knowledge. Once partitioned, availability techniques may be applied to each domain independently. We argue such an approach is necessary to deal with the geographic distribution of system components imposed by the nature of the Internet and maps well onto real e-commerce deployments.

Subscription propagation and content-based routing with delivery guarantees

Subscription propagation enables efficient content-based routing in publish/subscribe systems and is a challenging problem when it is required to support reliable delivery in networks with redundant routes. We have designed a generic model and a highly-asynchronous algorithm accomplishing these goals. Existing algorithms can be interpreted as different encodings and optimizations of the generic algorithm and hence their correctness can be derived from the generic algorithm.