Lanyue Lu

A Study of Linux File System Evolution

We conduct a comprehensive study of file-system code evolution. By analyzing eight years of Linux file-system changes across 5079 patches, we derive numerous new (and sometimes surprising) insights into the file-system development process; our results should be useful for both the development of file systems themselves as well as the improvement of bug-finding tools.

WiscKey: Separating Keys from Values in SSD-Conscious Storage

We present WiscKey, a persistent LSM-tree-based key-value store with a performance-oriented data layout that separates keys from values to minimize I/O amplification. The design of WiscKey is highly SSD optimized, leveraging both the sequential and random performance characteristics of the device. We demonstrate the advantages of WiscKey with both microbenchmarks and YCSB workloads. Microbenchmark results show that WiscKey is 2.5×-111× faster than LevelDB for loading a database and 1.6×-14× faster for random lookups. WiscKey is faster than both LevelDB and RocksDB in all six YCSB workloads.

DiskGroup: Energy Efficient Disk Layout for RAID1 Systems

Energy consumption is becoming an increasingly important issue in storage systems, especially for high performance data centers and network servers. In this paper, we introduce a family of energy-efficient disk layouts that generalize the data mirroring of a conventional RAID 1 system. The scheme called DiskGroup distributes the workload between the primary disks and secondary disks based on the characteristics of the workload. We develop an analytic model to explore the design space and compute the estimated energy savings and performance as a function of workload characteristics. The analysis shows the potential for significant energy savings over simple RAID1 data mirroring.

Graduated QoS by Decomposing Bursts: Don't Let the Tail Wag Your Server

The growing popularity of hosted storage services and shared storage infrastructure in data centers is driving the recent interest in resource management and QoS in storage systems. The bursty nature of storage workloads raises significant performance and provisioning challenges, leading to increased infrastructure, management, and energy costs. We present a novel dynamic workload shaping framework to handle bursty workloads, where the arrival stream is dynamically decomposed to isolate its bursts, and then rescheduled to exploit available slack. We show how decomposition reduces the server capacity requirements dramatically while affecting QoS guarantees minimally. We present an optimal decomposition algorithm RTT and a recombination algorithm Miser, and show the benefits of the approach by performance evaluation using several storage traces.

Application Crash Consistency and Performance with CCFS

Recent research has shown that applications often incorrectly implement crash consistency. We present the Crash-Consistent File System (ccfs), a file system that improves the correctness of application-level crash consistency protocols while maintaining high performance. A key idea in ccfs is the abstraction of a stream. Within a stream, updates are committed in program order, improving correctness; across streams, there are no ordering restrictions, enabling scheduling flexibility and high performance. We empirically demonstrate that applications running atop ccfs achieve high levels of crash consistency. Further, we show that ccfs performance under standard file-system benchmarks is excellent, in the worst case on par with the highest performing modes of Linux ext4, and in some cases notably better. Overall, we demonstrate that both application correctness and high performance can be realized in a modern file system.

Decomposing Workload Bursts for Efficient Storage Resource Management

The growing popularity of hosted storage services and shared storage infrastructure in data centers is driving the recent interest in resource management and QoS in storage systems. The bursty nature of storage workloads raises significant performance and provisioning challenges, leading to increased resource requirements, management costs, and energy consumption. We present a novel workload shaping framework to handle bursty workloads, where the arrival stream is dynamically decomposed to isolate its bursts, and then rescheduled to exploit available slack. We show how decomposition reduces the server capacity requirements and power consumption significantly, while affecting QoS guarantees minimally. We present an optimal decomposition algorithm RTT and a recombination algorithm Miser, and show the benefits of the approach by evaluating the performance of several storage workloads using both simulation and Linux implementation.

Workload decomposition for QoS in hosted storage services

The growing popularity of hosted storage services and shared storage infrastructure in data centers is driving the recent interest in performance isolation and QoS in storage systems. Due to the bursty nature of storage workloads, meeting the traditional response-time Service Level Agreements requires significant over provisioning of the server capacity. We present a graduated, distribution-based QoS specification for storage servers that provides cost benefits over traditional QoS models. Our method RTT partitions the workload to minimize the capacity required to meet response time requirements of any specified fraction of the requests.

CARP: Handling Silent Data Errors and Site Failures in an Integrated Program and Storage Replication Mechanism

This paper presents CARP, an integrated program and storage replication solution. CARP extends program replication systems which do not currently address storage errors, builds upon a record-and-replay scheme that handles nondeterminism in program execution, and uses a scheme based on recorded program state and I/O logs to enable efficient detection of silent data errors and efficient recovery from such errors. CARP is designed to be transparent to applications with minimal run-time impact and is general enough to be implemented on commodity machines. We implemented CARP as a prototype on the Linux operating system and conducted extensive sensitivity analysis of its overhead with different application profiles and system parameters. In particular, we evaluated CARP with standard unmodified email, database, and web server benchmarks and showed that it imposes acceptable overhead while providing sub-second program state recovery times on detecting a silent data error.

ZoneFS: Stripe remodeling in cloud data centers

Cloud data centers will contain tens of thousands of servers with massive aggregate bandwidth requirements for generating, accessing, and analyzing immense amounts of data. The I/O requirements of the myriad applications that these data centers must support run the gamut from extreme IOPS intensive to extreme bandwidth intensive. Delivering high performance with unreliable commodity hardware for this range of workloads is truly a grand challenge. ZoneFS is a parallel file system that targets cloud data center infrastructures built up of commodity network switches. ZoneFS employs a highly-available and flexible storage architecture that divides a cluster switch hierarchy into zones and stripes data across servers and disks to maximize aggregate I/O throughput and avoid storage server hotspots. In this paper, we present the overall design and implementation of ZoneFS and evaluate its key features with several cloud computing workloads. Our experimental results show that ZoneFS can improve application runtime performance by up to 76% over standard parallel file systems and by up to 85% over Internet-scale file systems.