Leigh Stoller

An integrated experimental environment for distributed systems and networks

Three experimental environments traditionally support network and distributed systems research: network emulators, network simulators, and live networks. The continued use of multiple approaches highlights both the value and inadequacy of each. Netbed, a descendant of Emulab, provides an experimentation facility that integrates these approaches, allowing researchers to configure and access networks composed of emulated, simulated, and wide-area nodes and links. Netbeds primary goals are ease of use, control, and realism, achieved through consistent use of virtualization and abstraction.By providing operating system-like services, such as resource allocation and scheduling, and by virtualizing heterogeneous resources, Netbed acts as a virtual machine for network experimentation. This paper presents Netbeds overall design and implementation and demonstrates its ability to improve experimental automation and efficiency. These, in turn, lead to new methods of experimentation, including automated parameter-space studies within emulation and straightforward comparisons of simulated, emulated, and wide-area scenarios.

Impulse: building a smarter memory controller

Impulse is a new memory system architecture that adds two important features to a traditional memory controller. First, Impulse supports application-specific optimizations through configurable physical address remapping. By remapping physical addresses, applications control how their data is accessed and cached, improving their cache and bus utilization. Second, Impulse supports prefetching at the memory controller, which can hide much of the latency of DRAM accesses. In this paper we describe the design of the Impulse architecture, and show how an Impulse memory system can be used to improve the performance of memory-bound programs. For the NAS conjugate gradient benchmark, Impulse improves performance by 67%. Because it requires no modification to processor, cache, or bus designs, Impulse can be adopted in conventional systems. In addition to scientific applications, we expect that Impulse will benefit regularly strided memory-bound applications of commercial importance, such as database and multimedia programs.

Mobile Emulab: A Robotic Wireless and Sensor Network Testbed

Simulation has been the dominant research method- ology in wireless and sensor networking. When mobility is added, real-world experimentation is especially rare. However, it is becoming clear that simulation models do not sufficiently capture radio and sensor irregularity in a complex, real-world environment, especially indoors. Unfortunately, the high labor and equipment costs of truly mobile experimental infrastructure present high barriers to such experimentation. We describe our experience in creating a testbed to lower those barriers. We have extended the Emulab network testbed software to provide the first remotely-accessible mobile wireless and sensor testbed. Robots carry motes and single board computers through a fixed indoor field of sensor-equipped motes, all running the users selected software. In real-time, interactively or driven by a script, remote users can position the robots, control all the computers and network interfaces, run arbitrary programs, and log data. Our mobile testbed provides simple path planning, a vision-based tracking system accurate to 1 cm, live maps, and webcams. Precise positioning and automation allow quick and painless evaluation of location and mobility effects on wireless protocols, location algorithms, and sensor-driven applications. The system is robust enough that it is deployed for public use. We present the design and implementation of our mobile testbed, evaluate key aspects of its performance, and describe a few experiments demonstrating its generality and power.

Making distributed shared memory simple, yet efficient

Recent research on distributed shared memory (DSM) has focussed on improving performance by reducing the communication overhead of DSM. Features added include lazy release consistency based coherence protocols and new interfaces that give programmers the ability to hand tune communication. These features have increased DSM performance at the expense of requiring increasingly complex DSM systems or increasingly cumbersome programming. They have also increased the computation overhead of DSM, which has partially offset the communication related performance gains. We chose to implement a simple DSM system, Quarks, with an eye towards hiding most computation overhead while using a very low latency transport layer to reduce the effect of communication overhead. The resulting performance is comparable to that of far more complex DSM systems, such as Treadmarks and Cashmere.

Designing a federated testbed as a distributed system

Traditionally, testbeds for networking and systems research have been stand-alone facilities: each is owned and operated by a single administrative entity, and is intended to be used independently of other testbeds. However, this isolated facility model is at odds with researchers’ ever-increasing needs for experiments at larger scale and with a broader diversity of network technologies. The research community will be much better served by a federated model. In this model, each federated testbed maintains its own autonomy and unique strengths, but all federates work together to make their resources available under a common framework.

FLEX: a tool for building efficient and flexible systems

Modern operating systems must support a wide variety of services for a diverse set of users. Designers of these systems face a tradeoff between functionality and performance. Systems like Mach provide a set of general abstractions and attempt to handle every situation, which can lead to poor performance for common cases. Other systems, such as Unix, provide a small set of abstractions that can be made very efficient, at the expense of functionality. We are implementing a flexible system building tool, FLEX, that allows us to support a powerful operating systems interface efficiently by constructing specialized module implementations at runtime. FLEX improves the performance of existing systems by optimizing interprocess communications paths and relocating servers and clients to reduce communications overhead. These facilities improve the performance of Unix system calls on Mach from 20-400%. Furthermore, FLEX can dynamically extend the kernel in a controlled fashion, which gives user programs access to privileged data and devices not envisioned by the original operating system implementor.<<ETX>>

Impulsec Memory system support for scientific applications

Impulse is a new memory system architecture that adds two important features to a traditional memory controller. First, Impulse supports application-specific optimizations through configurable physical address remapping. By remapping physical addresses, applications control how their data is accessed and cached, improving their cache and bus utilization. Second, Impulse supports prefetching at the memory controller, which can hide much of the latency of DRAM accesses. Because it requires no modification to processor, cache, or bus designs, Impulse can be adopted in conventional systems. In this paper we describe the design of the Impulse architecture, and show how an Impulse memory system can improve the performance of memory-bound scientific applications. For instance, Impulse decreases the running time of the NAS conjugate gradient benchmark by 67%. We expect that Impulse will also benefit regularly strided, memory-bound applications of commercial importance, such as database and multimedia programs.

Memory System Support for Irregular Applications

Because irregular applications have unpredictable memory access patterns, their performance is dominated by memory behavior. The Impulse configurable memory controller will enable significant performance improvements for irregular applications, because it can be configured to optimize memory accesses on an application-by-application basis. In this paper we describe the optimizations that the Impulse controller supports for sparse matrix-vector product, an important computational kernel, and outline the transformations that the compiler and runtime system must perform to exploit these optimizations.

Emulab's wireless sensor net testbed: true mobility, location precision, and remote access

We have extended the well-known Emulab network testbed software to support both fixed and mobile wireless sensor devices. Mobility is achieved through remotely-controlled robots. We have deployed this software in public production use for the research and education communities. The current temporary testbed is in a 60 square meter indoor area. It contains six robots and 25 fixed Mica2 motes with serial programming boards, 10 of which also have full sensor boards. The robots carry an Intel Stargate with an X-Scale 400MHz CPU running Linux, an 802.11a/b/g wireless Ethernet card, and a Mica2. A much larger testbed is contemplated. Robot motion can be scripted using the ns language or interactively controlled from a Java applet Webcam views integrated into the applet ease remote access. A high-precision (1cm) localization system provides precise positions of robots (and thus wireless antennae). Users have full control over the wireless devices on each robot, and can install custom software on the Stargate and mote. To provide users with precise, real-time robot control, we extended the core of Emulab with three new components. The robot control daemon, robotd, maneuvers robots to user-specified positions based on input from visiond, and

Netbed: an integrated experimental environment

The diverse requirements of network and distributed systems research are not well met by any single experimental environment. Competing approaches remain popular because each offers different levels of ease of use , control, andrealism. These include packet-level discrete event simulation, live network experimentation, and emulation, which subjects real hardware, protocols, and workloads to a synthetic network environment. Netbed offers a new alternative. It is software that, when deployed on local and wide-area machines, provides a platform for research, education, or development in distributed systems and networks. Netbed’s primary contribution is the seamless integrationof the above disparate techniques into a common architectural framework that preserves the control and ease of use of simulation, without sacrificing the realism of emulation and live network experimentation. This framework provides common abstractions, namespaces, services, and user interfaces to all three environments. Netbed’s integration allows tools such as topology and traffic generators that were originally targeted for one domain to apply to all. Netbed leverages and extends its ancestor, Emulab, which focused on making emulationas easy to use and control as simulation. Our results show that Emulab speeds up experiment configuration on a cluster by two orders of magnitude, and through timeand space-sharing, improves cluster utilization by another two orders of magnitude. Architecture: Netbed revolves around interacting state machines, monitored by a state management daemon. The central state machine is the “experiment.” Subsidiary state machines include those handling node allocation, configuration, and disk reloading. This approach copes well with the reliability challenges of large-scale distributed systems, composed of often unstable commodity hardware. The state daemon catches illegal or tardy state transitions. For example, if a node hangs while rebooting, the state daemon times out and attempts an alternate reboot mechanism. An experiment is generated from a user’s n script or via a GUI, resulting in hard state in Netbed’s relational database. It represents a network configuration, such as switch VLAN mappings or IP tunnels, and node configurations, including OS images. A node’s local disk is currently considered soft state; this allows an experiment to be “swapped out” and later regenerated from the database onto other hardware. The database provides a single namespace and abstractions for heterogeneous resources. The resulting resourcetransparency enables simulated, emulated, live,