Matthew Wachs

Stardust: tracking activity in a distributed storage system

Performance monitoring in most distributed systems provides minimal guidance for tuning, problem diagnosis, and decision making. Stardust is a monitoring infrastructure that replaces traditional performance counters with end-to-end traces of requests and allows for efficient querying of performance metrics. Such traces better inform key administrative performance challenges by enabling, for example, extraction of per-workload, per-resource demand information and per-workload latency graphs. This paper reports on our experience building and using end-to-end tracing as an on-line monitoring tool in a distributed storage system. Using diverse system workloads and scenarios, we show that such fine-grained tracing can be made efficient (less than 6% overhead) and is useful for on- and off-line analysis of system behavior. These experiences make a case for having other systems incorporate such an instrumentation framework.

Modeling the relative fitness of storage

Relative fitness is a new black-box approach to modeling the performance of storage devices. In contrast with an absolute model that predicts the performance of a workload on a given storage device, a relative fitness model predicts performance differences between a pair of devices. There are two primary advantages to this approach. First, because are lative fitness model is constructed for a device pair, the application-device feedback of a closed workload can be captured (e.g., how the I/O arrival rate changes as the workload moves from device A to device B). Second, a relative fitness model allows performance and resource utilization to be used in place of workload characteristics. This is beneficial when workload characteristics are difficult to obtain or concisely express (e.g., rather than describe the spatio-temporal characteristics of a workload, one could use the observed cache behavior of device A to help predict the performance of B. This paper describes the steps necessary to build a relative fitness model, with an approach that is general enough to be used with any black-box modeling technique. We compare relative fitness models and absolute models across a variety of workloads and storage devices. On average, relative fitness models predict bandwidth and throughput within 10-20% and can reduce prediction error by as much as a factor of two when compared to absolute models.

Incremental algorithm for updating betweenness centrality in dynamically growing networks

The increasing availability of dynamically growing digital data that can be used for extracting social networks has led to an upsurge of interest in the analysis of dynamic social networks. One key aspect of social network analysis is to understand the central nodes in a network. However, dynamic calculation of centrality values for rapidly growing networks might be unfeasibly expensive, especially if it involves recalculation from scratch for each time period. This paper proposes an incremental algorithm that effectively updates betweenness centralities of nodes in dynamic social networks while avoiding re-computations by exploiting information from earlier computations. Our performance results suggest that our incremental betweenness algorithm can achieve substantial performance speedup, on the order of thousands of times, over the state of the art, including the best-performing non-incremental betweenness algorithm and a recently proposed betweenness update algorithm.

Co-scheduling of Disk Head Time in Cluster-Based Storage

Disk time slicing is a promising technique for storage performance insulation. To work with cluster-based storage, however, time slices associated with striped data must be co-scheduled on the corresponding servers. This paper describes algorithms for determining global time slice schedules and mechanisms for coordinating the independent server activities. Experiments with a prototype show that, combined, they can provide performance insulation for workloads sharing a storage cluster -- each workload realizes a configured minimum efficiency within its time slices regardless of the activities of the other workloads.

Relative fitness modeling

Relative fitness is a new approach to modeling the performance of storage devices (e.g., disks and RAID arrays). In contrast to a conventional model, which predicts the performance of an applications I/O on a given device, a relative fitness model predicts performance differences between devices. The result is significantly more accurate predictions.

File system virtual appliances: Portable file system implementations

File system virtual appliances (FSVAs) address the portability headaches that plague file system (FS) developers. By packaging their FS implementation in a virtual machine (VM), separate from the VM that runs user applications, they can avoid the need to port the file system to each operating system (OS) and OS version. A small FS-agnostic proxy, maintained by the core OS developers, connects the FSVA to whatever OS the user chooses. This article describes an FSVA design that maintains FS semantics for unmodified FS implementations and provides desired OS and virtualization features, such as a unified buffer cache and VM migration. Evaluation of prototype FSVA implementations in Linux and NetBSD, using Xen as the virtual machine manager (VMM), demonstrates that the FSVA architecture is efficient, FS-agnostic, and able to insulate file system implementations from OS differences that would otherwise require explicit porting.