John J. Hudak

Development of a benchmark to measure system robustness

An initial attempt at the development of a set of benchmarks to gauge a systems robustness as measured by its ability to tolerate errors is presented. Due to the large domain of system components whose intolerance to errors can lead to system failure, several primitive benchmarks that can be combined into a robustness benchmark suite are presented. Each primitive benchmark targets a system functionality and measure its behavior given erroneous inputs. Four primitive benchmarks have been implemented in this initial effort. They target the file management system, memory access, user application, and the C library functions. The motivation and experimental results of each of these primitive benchmarks are presented in detail followed by an analysis of the results. A methodology to combine the primitive benchmarks to form an overall robustness figure is presented. A list of additional primitive benchmarks is suggested.

Evaluation and comparison of fault-tolerant software techniques

Four implementations of fault-tolerant software techniques are evaluated with respect to hardware and design faults. Project participants were divided into four groups, each of which developed fault-tolerant software based on a common specification. Each group applied one of the following techniques: N-version programming, recovery block, concurrent error-detection, and algorithm-based fault tolerance. Independent testing and modeling groups analyzed the software. The testing group subjected it to simulated design and hardware faults. The data were then mapped into a discrete-time Markov model developed by the modeling group. The effectiveness of each technique with respect to availability, correctness, and time to failure given an error, as shown by the model, is contrasted with measured data. The model is analyzed with respect to additional figures of merit identified during the modeling process, and the techniques are ranked using an application taxonomy. >