Jay J. Wylie

Flat XOR-based erasure codes in storage systems: Constructions, efficient recovery, and tradeoffs

Large scale storage systems require multi-disk fault tolerant erasure codes. Replication and RAID extensions that protect against two- and three-disk failures offer a stark tradeoff between how much data must be stored, and how much data must be read to recover a failed disk. Flat XOR-codes-erasure codes in which parity disks are calculated as the XOR of some subset of data disks-offer a tradeoff between these extremes. In this paper, we describe constructions of two novel flat XOR-code, Stepped Combination and HD-Combination codes. We describe an algorithm for flat XOR-codes that enumerates recovery equations, i.e., sets of disks that can recover a failed disk. We also describe two algorithms for flat XOR-codes that generate recovery schedules, i.e., sets of recovery equations that can be used in concert to achieve efficient recovery. Finally, we analyze the key storage properties of many flat XOR-codes and of MDS codes such as replication and RAID 6 to show the cost-benefit tradeoff gap that flat XOR-codes can fill.

Determining Fault Tolerance of XOR-Based Erasure Codes Efficiently

We propose a new fault tolerance metric for XOR-based erasure codes: the minimal erasures list (MEL). A minimal erasure is a set of erasures that leads to irrecoverable data loss and in which every erasure is necessary and sufficient for this to be so. The MEL is the enumeration of all minimal erasures. An XOR-based erasure code has an irregular structure that may permit it to tolerate faults at and beyond its Hamming distance. The MEL completely describes the fault tolerance of an XOR-based erasure code at and beyond its Hamming distance; it is therefore a useful metric for comparing the fault tolerance of such codes. We also propose an algorithm that efficiently determines the MEL of an erasure code. This algorithm uses the structure of the erasure code to efficiently determine the MEL. We show that, in practice, the number of minimal erasures for a given code is much less than the total number of sets of erasures that lead to data loss: in our empirical results for one corpus of codes, there were over 80 times fewer minimal erasures. We use the proposed algorithm to identify the most fault tolerant XOR-based erasure code for all possible systematic erasure codes with up to seven data symbols and up to seven parity symbols.

Efficient eventual consistency in Pahoehoe, an erasure-coded key-blob archive

Cloud computing demands cheap, always-on, and reliable storage. We describe Pahoehoe, a key-value cloud storage system we designed to store large objects cost-effectively with high availability. Pahoehoe stores objects across multiple data centers and provides eventual consistency so to be available during network partitions. Pahoehoe uses erasure codes to store objects with high reliability at low cost. Its use of erasure codes distinguishes Pahoehoe from other cloud storage systems, and presents a challenge for efficiently providing eventual consistency. We describe Pahoehoes put, get, and convergence protocols—convergence being the decentralized protocol that ensures eventual consistency. We use simulated executions of Pahoehoe to evaluate the efficiency of convergence, in terms of message count and message bytes sent, for failure-free and expected failure scenarios (e.g., partitions and server unavailability). We describe and evaluate optimizations to the naïve convergence protocol that reduce the cost of convergence in all scenarios.

Reliability of flat XOR-based erasure codes on heterogeneous devices

XOR-based erasure codes are a computationally-efficient means of generating redundancy in storage systems. Some such erasure codes provide irregular fault tolerance: some subsets of failed storage devices of a given size lead to data loss, whereas other subsets of failed storage devices of the same size are tolerated. Many storage systems are composed of heterogeneous devices that exhibit different failure and recovery rates, in which different placements- mappings of erasure-coded symbols to storage devices-of a flat XOR-based erasure code lead to different reliabilities. We have developed redundancy placement algorithms that utilize the structure of flat XOR-based erasure codes and a simple analytic model to determine placements that maximize reliability. Simulation studies validate the utility of the simple analytic reliability model and the efficacy of the redundancy placement algorithms.

Client-centric benchmarking of eventual consistency for cloud storage systems

Eventually-consistent key-value storage systems sacrifice the ACID semantics of conventional databases to achieve superior latency and availability. However, this means that client applications, and hence end-users, can be exposed to stale data. The degree of staleness observed depends on various tuning knobs set by application developers (customers of key-value stores) and system administrators (providers of key-value stores). Both parties must be cognizant of how these tuning knobs affect the consistency observed by client applications in the interest of both providing the best end-user experience and maximizing revenues for storage providers. Quantifying consistency in a meaningful way is a critical step toward both understanding what clients actually observe, and supporting consistency-aware service level agreements (SLAs) in next generation storage systems. This paper proposes a novel consistency metric called Gamma that captures client-observed consistency. This metric provides quantitative answers to questions regarding observed consistency anomalies, such as how often they occur and how bad they are when they do occur. We argue that Gamma is more useful and accurate than existing metrics. We also apply Gamma to benchmark the popular Cassandra key-value store. Our experiments demonstrate that Gamma is sensitive to both the workload and client-level tuning knobs, and is preferable to existing techniques which focus on worst-case behavior.

Prato: Databases on Demand

Database configuration can be a daunting task as database administrators are often presented with a myriad of configuration options that are difficult to sift through. Prato, a project at HP Labs, is a prototype of a self-managing DBMS service provider that eases this burden by using economic incentives to guide automated DBMS setup and management. Prato offers customers private, virtual, DBMS appliances that can each be sized up to several hundred nodes, and made available on demand, in just a few minutes.

Applying idealized lower-bound runtime models to understand inefficiencies in data-intensive computing

“Data-intensive scalable computing” (DISC) refers to a rapidly growing style of computing characterized by its reliance on large and expanding datasets [3]. Driven by the desire and capability to extract insight from such datasets, DISC is quickly emerging as a major activity of many organizations. Map-reduce style programming frameworks such as MapReduce [4] and Hadoop [1] support DISC activities by providing abstractions and frameworks to more easily scale data-parallel computations over commodity machines. In the pursuit of scale, popular map-reduce frameworks neglect efficiency as an important metric. Anecdotal experiences indicate that they neither achieve balance nor full goodput of hardware resources, effectively wasting a large fraction of the computers over which jobs are scaled. If these inefficiencies are real, the same work could be completed at much lower costs. An ideal run would provide maximum scalability for a given computation without wasting resources. Given the widespread use and scale of DISC systems, it is important that we move closer to frameworks that are “hardwareefficient,” where the framework provides sufficient parallelism to keep the bottleneck resource fully utilized and makes good use of all I/O components. An important first step is to understand the degree, characteristics, and causes of inefficiency. We have a simple model that predicts the idealized lower-bound runtime of a map-reduce workload by assuming an even data distribution, that data is perfectly pipelined through sequential operations, and that the underlying I/O resources are utilized at their full bandwidths whenever applicable. The model’s input parameters describe basic characteristics of the job (e.g., amount of input data), of the hardware (e.g., per-node disk and network throughputs), and of the framework configuration (e.g., replication factor). The output is the idealized runtime. The goal of the model is not to accurately predict the runtime of a job on any given system, but to indicate what the runtime theoretically should be. To focus the evaluation on the efficiency of the programming framework, and not the entire software stack, mea-

Finding the most fault-tolerant flat XOR-based erasure codes for storage systems

We describe the techniques we developed to efficiently find the most fault-tolerant flat XOR-based erasure codes for storage systems. These techniques substantially reduce the search space for finding fault-tolerant codes (e.g., by a factor of over 52 trillion in one case). This reduction in the search space has allowed us to find the most fault-tolerant codes for larger codes than was previously thought feasible. The result of our effort to find the most fault-tolerant flat XOR-based erasure codes for storage systems has yielded a corpus of 49,215 erasure codes that we are making public.

Computer systems research at HP labs

• Information explosion: Acquiring, analyzing and delivering the right information to individuals and businesses so they can act on it. • Dynamic cloud services: Developing web platforms and cloud services that are dynamically personalized based on your location, preferences, calendar and communities. • Content transformation: Enabling the fluid transformation of content from analog to digital, from device to device, and from digital content to physical products. • Intelligent infrastructure: Designing smarter, more-secure devices, networks and scalable architectures that work together to connect individuals and businesses to rich, dynamic content and services. • Sustainability: Creating technologies, IT infrastructure, and new business models for a lower-carbon economy, to save money and leave a lighter footprint on the environment.