John P. J. Kelly

Implementing design diversity to achieve fault tolerance

The software faults that are particularly significant in a real-time concurrent system are identified, and the use of design diversity to prevent their occurrence is examined. Two approaches to enforced diversity, recovery-block software and multiversion software, are discussed. The recovery-block scheme combines N diverse software versions arranged (conceptually, at least) in sequential order, although the versions may also be organized to execute concurrently. The multiversion-software approach excuses all N versions in parallel, taking advantage of the redundant processors likely to be available in any system that must tolerate hardware and software faults. Although different, both approaches require sufficiently diverse development environments and that faults in the specification do not lead to similar errors.<<ETX>>

A large scale second generation experiment in multi-version software: description and early results

The second-generation experiment is a large-scale empirical study of the development and operation of multiversion software systems that has engaged researchers at five universities and three research institutes. The authors present the history and current status of this experiment. The primary objective for the second generation experiments is an examination of multiple-version reliability improvement. Experimentation concerns have been focused on the development of multiversion software (MVS) systems, primarily design and testing issues, and the modeling and analysis of these systems. A preliminary analysis of the multiple software versions has been performed and is reported.<<ETX>>

Experiences in applying formal methods to the analysis of software and system requirements

In an effort to improve the quality of software and system requirements, formal methods (FM) is being investigated by NASA because evidence existed that FM is useful in creating consistent and verifiable specifications. This investigation of FM consists of trial projects that are used to gather data on FMs cost-effectiveness and to demonstrate this effectiveness to prospective users. These trial projects were specifically constructed to maximize the likelihood that requirements analysts will recognize the benefits of FM and integrate it into their existing approaches. A key aspect of these trial-projects is the make-up of the teams which influenced the planning, execution, and evaluation of results. Through these projects, much has been learned about the use of FM and its potential for being accepted as a viable way to improve requirements analysis.

Achieving dependability throughout the development process: a distributed software experiment

Distributed software engineering techniques and methods for improving the specification and testing phases are considered. To examine these issues, an experiment was performed using the design diversity approach in the specification, design, implementation, and testing of distributed software. In the experiment, three diverse formal specifications were used to produce multiple independent implementations of a distributed communication protocol in Ada. The problems encountered in building complex concurrent processing systems in Ada were also studied. Many pitfalls were discovered in mapping the formal specifications into Ada implementations. >

Analysis of faults detected in a large-scale multi-version software development experiment

In a multiversion software experiment, twenty programs were built to the same specification of an inertial navigation problem. The programs were then subjected to a three-phase testing and debugging process: an acceptance test, a certification test, and an operational test. Less than 20% of the faults discovered during the certification and operational testing were nonunique, i.e. the same or very similar faults would be found in more than one program. However, some of these common faults spanned as many as half of the versions. Faults discovered during the certification testing were due to specification errors and ambiguities, inadequate programmer background knowledge, insufficient programming experience, incomplete analysis, and insufficient acceptance testing. Faults discovered during the operational testing were of a more subtle nature, and were mostly due to various programmer knowledge defects and incomplete analysis errors. Techniques that might have prevented the observed faults are discussed.<<ETX>>

Experiences with Estelle, LOTOS and SDL: a protocol implementation experiment

Abstract A controlled experiment was conducted in which six protocol implementations were developed. They were derived by integrating two protocols specified with the ISO and CCITT Formal Description Techniques (FDTs) Estelle, LOTOS, and SDL. The implementations were written in the Ada programming language by six independent programming teams (two teams worked from each FDT). Experience with the specifications is presented: in particular, how readable and precise the teams found their specifications and how they mapped the specifications into Ada constructs. Results are also presented concerning the number and classification of implementation faults. Experience using Ada, particularly its concurrency features and the efficiency of the implementation, is also included.

Techniques for building dependable distributed systems: multi-version software testing

To investigate the effectiveness of serializable back-to-back testing and other issues in multiversion software systems, an experiment was performed. The authors discuss the use of multiple implementations for fault prevention throughout development, particularly during the testing phase. The specifications chosen were written in languages that meet industrial standards. The application is a communication protocol based on the Open Systems Interconnection (OSI) layered model adopted by the International Organization for Standardization (ISO) in 1979. The OSI layered model is introduced, the generation of appropriate test cases is discussed, and the testing environment is presented. The serializable back-to-back testing paradigm is presented in detail, along with testing results.<<ETX>>

An Empirical Investigation of the Effect of Formal Specifications on Program Diversity

Formal specification languages are increasingly being employed as an aid in the design and implementation of highly reliable systems. Recent experimental evidence indicates that the syntax and semantics associated with a formal specification language can have a large effect on the subsequent program version. This paper analyses the effect formal specification languages have on program development by examining nine diverse versions of a communication protocol created using three different formal specification languages.