Wee Teck Ng

The systematic improvement of fault tolerance in the Rio file cache

Fault injection is typically used to characterize failures and to validate and compare fault-tolerant mechanisms. However fault injection is rarely used for all these purposes to guide the design and implementation of a fault tolerant system. We present a systematic and quantitative approach for using software-implemented fault injection to guide the design and implementation of a fault-tolerant system. Our system design goal is to build a write-back file cache on Intel PCs that is as reliable as a write-through file cache. We follow an iterative approach to improve robustness in the presence of operating system errors. In each iteration, we measure the reliability of the system, analyze the fault symptoms that lead to data con option, and apply fault-tolerant mechanisms that address the fault symptoms. Our initial system is 13 times less reliable than a write-through file cache. The result of several iterations is a design that is both more reliable (1.9% vs. 3.1% corruption rate) and 5-9 times as fast as a write-through file cache.

Integrating reliable memory in databases

Abstract. Recent results in the Rio project at the University of Michigan show that it is possible to create an area of main memory that is as safe as disk from operating system crashes. This paper explores how to integrate the reliable memory provided by the Rio file cache into a database system. Prior studies have analyzed the performance benefits of reliable memory; we focus instead on how different designs affect reliability. We propose three designs for integrating reliable memory into databases: non-persistent database buffer cache, persistent database buffer cache, and persistent database buffer cache with protection. Non-persistent buffer caches use an I/O interface to reliable memory and require the fewest modifications to existing databases. However, they waste memory capacity and bandwidth due to double buffering. Persistent buffer caches use a memory interface to reliable memory by mapping it into the database address space. This places reliable memory under complete database control and eliminates double buffering, but it may expose the buffer cache to database errors. Our third design reduces this exposure by write protecting the buffer pages. Extensive fault tests show that mapping reliable memory into the database address space does not significantly hurt reliability. This is because wild stores rarely touch dirty, committed pages written by previous transactions. As a result, we believe that databases should use a memory interface to reliable memory.

The design and verification of the Rio file cache

Todays file systems are limited in speed and reliability by memorys vulnerability to operating system crashes. Because memory is viewed as unsafe, systems periodically write modified file data back to disk. These extra disk writes lower system performance and the delay period before data is safe lowers reliability. The goal of the Rio (RAM I/O) file cache is to make ordinary main memory safe for persistent storage by enabling memory to survive operating system crashes. Reliable main memory enables the Rio file cache to be as reliable as a write-through file cache, where every write is safe instantly, and as fast as a pure write-back file cache, with no reliability-induced writes to disk. This paper describes the systematic, quantitative process we used to design and verify the Rio file cache on Intel PCs running FreeBSD and the reliability and performance of the resulting system.

Comparing disk and memory's resistance to operating system crashes

Memory is commonly viewed as an unreliable place to store permanent data (files) because it is perceived to be vulnerable to system crashes. Yet despite all the negative implications of memorys unreliability, no data exists that quantifies how vulnerable memory actually is to system crashes. This paper quantitatively compares the vulnerability of disk and memory to operating system crashes. We use software fault injection to induce a wide variety of operating system crashes in DEC Alpha workstations running Digital Unix, ranging from bit errors in the kernel stack to deleting branch instructions to C-level allocation management errors. We find that files on disk are rarely corrupted (1.1% corruption rate), which agrees with our intuition. We also find that, surprisingly files in memory are nearly as safe as files on disk. Only 10 of the 650 crashes we observed (1.5%) corrupt any files in memory. Our data contradicts the common assumption that operating system crashes often corrupt files in memory and suggests that memory can be used to store permanent data rather than needing to write it back to disk.

Obtaining high performance for storage outsourcing

The viability of storage outsourcing is critically dependent on the access performance of remote storage. We study this issue by measuring the behavior of a broad variety of I/O-intensive benchmarks as they access remote storage over an IP network. We measure the effect of network latencies that correspond to distances ranging from a local neighborhood to halfway across a continent. We then measure the effect of latency-hiding mechanisms. Our results indicate that, in many cases, the adverse effects of network delay can be rendered inconsequential by clever file system and operating system techniques.