Michael B. Jones

An evaluation of scalable application-level multicast built using peer-to-peer overlays

Structured peer-to-peer overlay networks such as CAN, Chord, Pastry, and Tapestry can be used to implement Internet-scale application-level multicast. There are two general approaches to accomplishing this: tree building and flooding. This paper evaluates these two approaches using two different types of structured overlay: 1) overlays which use a form of generalized hypercube routing, e.g., Chord, Pastry and Tapestry, and 2) overlays which use a numerical distance metric to route through a Cartesian hyperspace, e.g., CAN. Pastry and CAN are chosen as the representatives of each type of overlay. To the best of our knowledge, this paper reports the first head-to-head comparison of CAN-style versus Pastry-style overlay networks, using multicast communication workloads running on an identical simulation infrastructure. The two approaches to multicast are independent of overlay network choice, and we provide a comparison of flooding versus tree-based multicast on both overlays. Results show that the tree-based approach consistently outperforms the flooding approach. Finally, for tree-based multicast, we show that Pastry provides better performance than CAN.

Interposition agents: transparently interposing user code at the system interface

Many contemporary operating systems utilize a system call interface between the operating system and its clients. Increasing numbers of systems are providing low-level mechanisms for intercepting and handling system calls in user code. Nonetheless, they typically provide no higher-level tools or abstractions for effectively utilizing these mechanisms. Using them has typically required reimplementation of a substantial portion of the system interface from scratch, making the use of such facilities unwieldy at best.This paper presents a toolkit that substantially increases the ease of interposing user code between clients and instances of the system interface by allowing such code to be written in terms of the high-level objects provided by this interface, rather than in terms of the intercepted system calls themselves. This toolkit helps enable new interposition agents to be written, many of which would not otherwise have been attempted.This toolkit has also been used to construct several agents including: system call tracing tools, file reference tracing tools, and customizable filesystem views. Examples of other agents that could be built include: protected environments for running untrusted binaries, logical devices implemented entirely in user space, transparent data compression and/or encryption agents, transactional software environments, and emulators for other operating system environments.

Herald: achieving a global event notification service

This paper presents the design philosophy and initial design decisions of Herald: a highly scalable global event notification system that is being designed and built at Microsoft Research. Herald is a distributed system designed to transparently scale in all respects, including numbers of subscribers and publishers, numbers of event subscription points, and event delivery rates. Event delivery can occur within a single machine, within a local network or Intranet, and throughout the Internet. Herald tries to take into account the lessons learned from the successes of both the Internet and the Web. Most notably, Herald is being designed, like the Internet, to operate correctly in the presence of numerous broken and disconnected components. The Herald service will be constructed as a set of protocols governing a federation of machines within cooperating but mutually suspicious domains of trust. Like the Web, Herald will try to avoid, to the extent possible, the maintenance of globally consistent state and will make failures part of the client-visible interface.

An overview of the Rialto real-time architecture

The goal of the Rialto project at Microsoft Research is to build a system architecture supporting coexisting independent real-time (and non-real-time) programs. Unlike traditional embedded-systems real-time environments, where timing and resource analysis among competing tasks can be done off-line, it is our goal to allow multiple independently authored real-time applications with varying timing and resource requirements to dynamically coexist and cooperate to share the limited physical resources available to them, as well as also coexisting with non-real-time applications.This paper gives an overview of the Rialto real-time architecture as it is implemented today and reports on some of the early results obtained. In particular, it describes the use of time constraints, activities, CPU and other resource reservation, and the system resource planner, and how they work together to achieve our goal of providing a flexible, dynamic real-time computing environment.

Modular real-time resource management in the Rialto operating system

This paper describes ongoing investigations into algorithms for modular distributed real-time resource management. These investigations are being conducted in the context of the Rialto operating system-an object-based real-time kernel and programming environment currently being developed within Microsoft Research. Some of the goals of this research include developing appropriate real-time programming abstractions to allow multiple independent real-time programs to dynamically coexist and share resources on the same hardware platforms. Use of these abstractions is intended both to allow individual applications to reason about their own resource requirements and for per-machine system resource planner applications to reason about and control resource allocations between potentially competing applications. The set of resources being managed is dynamically extensible, and may include remote resources in distributed environments. The local planner conducts resource negotiations with individual applications on behalf of the user, with the goal of maximizing the users perceived utility of the set of running applications with respect to resource allocations for those applications.

Design Rationale behind the Identity Metasystem Architecture

Many of the problems facing the Internet today stem from the lack of a widely deployed, easily understood, secure identity solution. Microsoft’s “InfoCard” project and the Identity Metasystem vision underlying it are aimed at filling this gap using technology all can adopt and solutions all can endorse, putting users in control of their identity interactions on the Internet. The design decisions presented in this paper are intended to result in a widely accepted, broadly applicable, inclusive, comprehensible, privacyenhancing, securityenhancing identity solution for the Internet. We present them and the rationale behind them to facilitate review of these design decisions by the security, privacy, and policy communities, so that people will better understand Microsoft’s implementations, and to help guide others when building interoperating implementations.

A simple and efficient implementation of a small database

This paper describes a technique for implementing the sort of small databases that frequently occur in the design of operating systems and distributed systems. We take advantage of the existence of very large virtual memories, and quite large real memories, to make the technique feasible. We maintain the database as a strongly typed data structure in virtual memory, record updates incrementally on disk in a log and occasionally make a checkpoint of the entire database. We recover from crashes by restoring the database from an old checkpoint then replaying the log. We use existing packages to convert between strongly typed data objects and their disk representations, and to communicate strongly typed data across the network (using remote procedure calls). Our memory is managed entirely by a general purpose allocator and garbage collector. This scheme has been used to implement a name server for a distributed system. The resulting implementation has the desirable property of being simultaneously simple, efficient and reliable.

Overlook: scalable name service on an overlay network

This paper indicates that a scalable fault-tolerant name service can be provided utilizing an overlay network and that such a name service can scale along a number of dimensions: it can be sized to support a large number of clients, it can allow large numbers of concurrent lookups on the same name or sets of names, and it can provide name lookup latencies measured in seconds. Furthermore, it can enable updates to be made pervasively visible in times typically measured in seconds for update rates of up to hundreds per second. We explain how many of these scaling properties for the name service are obtained by reusing some of the same mechanisms that allowed the underlying overlay network to scale. Finally, we observe that the overlay network is sensitive to bandwidth and CPU limitations.

Support for User-Centric Modular Real-Time Resource Management in the Rialto Operating System

This paper describes ongoing investigations into algorithms for user-centric modular distributed real-time resource management. These investigations are being conducted in the context of the Rialto operating system — an object-based real-time kernel and programming environment currently being developed within Microsoft Research.

Adaptive Real-Time Resource Management Supporting Modular Composition of Digital Multimedia Services

This paper describes ongoing investigations at Microsoft Research into algorithms for adaptive real-time scheduling and resource management. We are implementing a system that will allow sets of independently authored real-time applications to be scheduled, providing both initial resource negotiation and dynamic feedback to applications about their resource consumption. Instead of relying on human beings to pre-compute scheduling and resource requirements, applications will be able to dynamically adapt their run-time behavior to the resources available. Under overload conditions, user preferences and policies will be applied in order to bring the total resource requirements within available resources. This will permit chosen applications to continue executing perfectly even while others execute with dynamically renegotiated resource grants or are gracefully shut down, as per user preferences. We believe that this approach will lead to real-time scheduling and resource management algorithms that are appropriate for workstation and home multimedia environments.