Mike Hibler

An integrated experimental environment for distributed systems and networks

Three experimental environments traditionally support network and distributed systems research: network emulators, network simulators, and live networks. The continued use of multiple approaches highlights both the value and inadequacy of each. Netbed, a descendant of Emulab, provides an experimentation facility that integrates these approaches, allowing researchers to configure and access networks composed of emulated, simulated, and wide-area nodes and links. Netbeds primary goals are ease of use, control, and realism, achieved through consistent use of virtualization and abstraction.By providing operating system-like services, such as resource allocation and scheduling, and by virtualizing heterogeneous resources, Netbed acts as a virtual machine for network experimentation. This paper presents Netbeds overall design and implementation and demonstrates its ability to improve experimental automation and efficiency. These, in turn, lead to new methods of experimentation, including automated parameter-space studies within emulation and straightforward comparisons of simulated, emulated, and wide-area scenarios.

Microkernels meet recursive virtual machines

This paper describes a novel approach to providingmodular and extensible operating system functionality and encapsulated environments based on a synthesis of microkernel and virtual machine concepts. We have developed a software-based virtualizable architecture called Fluke that allows recursive virtual machines (virtual machines running on other virtual machines) to be implemented efficiently by a microkernel running on generic hardware. A complete virtual machine interface is provided at each level; efficiency derives from needing to implement only new functionality at each level. This infrastructure allows common OS functionality, such as process management, demand paging, fault tolerance, and debugging support, to be provided by cleanly modularized, independent, stackable virtual machine monitors, implemented as user processes. It can also provide uncommon or unique OS features, including the above features specialized for particular applications’ needs, virtual machines transparently distributed cross-node, or security monitors that allow arbitrary untrusted binaries to be executed safely. Our prototype implementation of this model indicates that it is practical to modularize operating systems this way. Some types of virtual machine layers impose almost no overhead at all, while others impose some overhead (typically 0–35%), but only on certain classes of applications.

An OS interface for active routers

This paper describes an operating system (OS) interface for active routers. This interface allows code loaded into active routers to access the routers memory, communication, and computational resources on behalf of different packet flows. In addition to motivating and describing the interface, the paper also reports our experiences implementing the interface in three different OS environments: Scout, the OSKit, and the esokernel.

Janos: a Java-oriented OS for active network nodes

Janos is an operating system for active network nodes whose primary focus is strong resource management and control of untrusted active applications written in Java. Janos includes the three major components of a Java-based active network operating system: the low-level NodeOS, a resource-aware Java Virtual Machine, and an active network protocol execution environment. Each of these components is separately usable. This article lays out the Janos design and its rationale.

User-level checkpointing through exportable kernel state

Checkpointing, process migration, and similar services need to have access not only to the memory of the constituent processes, but also to the complete state of all kernel provided objects (e.g., threads and ports) involved. Traditionally, a major stumbling block in these operations is acquiring and re-creating the state in the operating system. We have implemented a transparent user-mode checkpointer as an application on our Fluke microkernel. This microkernel consistently and cleanly supports the importing and exporting of fundamental kernel state safely to and from user applications. Implementing a transparent checkpointing facility with this sort of kernel support simplifies the implementation, and expands its flexibility and power.

Transparent checkpoints of closed distributed systems in Emulab

Emulab is a testbed for networked and distributed systems experimentation. Two guiding principles of its design are realism and control of experimentation. There is an inherent tension between these goals, however, and in some aspects of the testbeds design, Emulabs implementers favored realism over control. Thus, Emulab provides wide-ranging control over an experiments environment and initial conditions, but relatively little control over its execution--in particular, the ability to suspend, preempt, or replay the experiment. We have extended Emulab with a new means of control over experiment execution: the ability to cleanly checkpoint the execution of the set of nodes and networks that comprise an experiment. Conventional checkpoint mechanisms can easily degrade the fidelity of experiment results as a consequence of checkpoint downtimes, overheads of background state saving, and unintended distributed checkpoint synchronization effects. In this paper we demonstrate a checkpointing technique that is transparent with respect to the execution of the system under test, almost completely concealing the underlying checkpoint activity. Building on our checkpoint mechanism, we have implemented two powerful facilities for experiment execution control: the ability to preemptively swap-out experiments without losing their run-time state, and the ability to time-travel through the run of a system.

FLEX: a tool for building efficient and flexible systems

Modern operating systems must support a wide variety of services for a diverse set of users. Designers of these systems face a tradeoff between functionality and performance. Systems like Mach provide a set of general abstractions and attempt to handle every situation, which can lead to poor performance for common cases. Other systems, such as Unix, provide a small set of abstractions that can be made very efficient, at the expense of functionality. We are implementing a flexible system building tool, FLEX, that allows us to support a powerful operating systems interface efficiently by constructing specialized module implementations at runtime. FLEX improves the performance of existing systems by optimizing interprocess communications paths and relocating servers and clients to reduce communications overhead. These facilities improve the performance of Unix system calls on Mach from 20-400%. Furthermore, FLEX can dynamically extend the kernel in a controlled fashion, which gives user programs access to privileged data and devices not envisioned by the original operating system implementor.<<ETX>>

The persistent relevance of the local operating system to global applications

The growth and popularity of loosely-coupled distributed systems such as the World Wide Web and the touting of Java-based systems as the solution to the issues of software maintenance, flexibility, and security are changing the research emphasis away from traditional single node operating system issues. Apparently, the view is that traditional OS issues are either solved problems or minor problems. By contrast, we believe that building such vast distributed systems upon the fragile infrastructure provided by todays operating systems is analogous to building castles on sand. In this paper we outline the supporting arguments for these views and describe an OS design that supports secure encapsulation of the foreign processes that will be increasingly prevalent in tomorrows distributed systems.1

Using annotated interface definitions to optimize RPC

Abstract In RPC-based communication, it is useful to distinguish the RPC interface, which is the “network contract” between the client and the server, from the presentation, which is the “programmer’s contract” between the RPC stubs and the code that calls or is called by them. Presentation is usually a fixed function of the RPC interface, but some RPC systems, such as DCE and Concert, support the notion of a flexible presentation or endpoint modifier, allowing controlled modification of the behavior of the stubs on each side without affecting the contract between the client and the server. Up until now, the primary motivation for flexible presentation has been for programmer convenience and improved interoperability. However, we have found flexible presentation also to be useful for optimization of RPC, and in many cases necessary to achieving maximal performance without throwing out the RPC system and resorting to hand-coded stubs. In this paper we provide examples demonstrating this point for a number of different operating systems and IPC transport mechanisms, with RPC performance improvements ranging from 5% to an order of magnitude. In general, we observe that the more efficient the underlying IPC transport mechanism is, the more important it is for the RPC system to support flexible presentation, in order to avoid unnecessary user-space overhead.1

A3: An Environment for Self-Adaptive Diagnosis and Immunization of Novel Attacks

This paper describes an ongoing research effort aiming to use adaptation to defend individual applications against novel attacks. Application focused adaptive security spans adaptive use of security mechanisms in both the host and the network. The work presented in this paper is developing key infrastructure capabilities and supporting services including mandatory mediation of application I/O, record and replay of channel interaction, and VMI-based monitoring and analysis of execution that will facilitate replay-based diagnosis and patch derivation for attacks that succeed and go unnoticed until a known undesired condition manifests. After describing the basics, we present the results from our initial evaluation and outline the next steps.