Dolores R. Wallace

Software fault interactions and implications for software testing

Exhaustive testing of computer software is intractable, but empirical studies of software failures suggest that testing can in some cases be effectively exhaustive. We show that software failures in a variety of domains were caused by combinations of relatively few conditions. These results have important implications for testing. If all faults in a system can be triggered by a combination of n or fewer parameters, then testing all n-tuples of parameters is effectively equivalent to exhaustive testing, if software behavior is not dependent on complex event sequences and variables have a small set of discrete values.

Experimental models for validating technology

Experimentation helps determine the effectiveness of proposed theories and methods. However, computer science has not developed a concise taxonomy of methods for demonstrating the validity of new techniques. Experimentation is a crucial part of attribute evaluation and can help determine whether methods used in accordance with some theory during product development will result in software being as effective as necessary. By looking at multiple examples of technology validation, the authors develop a taxonomy for software engineering experimentation that describes twelve different experimental approaches.

Experimental validation in software engineering

Although experimentation is an accepted approach toward scientific validation in most scientific disciplines, it only recently has gained acceptance within the software development community. In this paper we discuss a 12-model classification scheme for performing experimentation within the software development domain. We evaluate over 600 published papers in the computer science literature and over one hundred papers from other scientific disciplines in order to determine: (1) how well the computer science community is succeeding at validating its theories, and (2) how computer science compares to other scientific disciplines. Published by Elsevier Science B.V.

FAILURE MODES IN MEDICAL DEVICE SOFTWARE: AN ANALYSIS OF 15 YEARS OF RECALL DATA

Most complex systems today contain software, and systems failures activated by software faults can provide lessons for software development practices and software quality assurance. This paper presents an analysis of software-related failures of medical devices that caused no death or injury but led to recalls by the manufacturers. The analysis categorizes the failures by their symptoms and faults, and discusses methods of preventing and detecting faults in each category. The nature of the faults provides lessons about the value of generally accepted quality practices for prevention and detection methods applied prior to system release. It also provides some insight into the need for formal requirements specification and for improved testing of complex hardware-software systems.

Software verification and validation: an overview

An explanation is given of software verification and validation (V&V) and how it fits in the development life cycle. How to apply V&V is also discussed. Evaluations of its effectiveness are summarized.<<ETX>>

Lessons from 342 medical device failures

Most complex systems today contain software, and systems failures activated by software faults can provide lessons for software development practices and software quality assurance. This paper presents an analysis of software-related failures of medical devices that caused no death or injury but led to recalls by the manufacturers. The analysis categorizes the failures by their symptoms and faults, and discusses methods of preventing and detecting faults in each category. The nature of the faults provides lessons about the value of generally accepted quality practices for prevention and detection methods applied prior to system release. It also provides some insight into the need for formal requirements specification and for improved testing of complex hardware-software systems.

Validation of object oriented software design with fault tree analysis

Software plays an increasing role in the safety critical systems. Increasing the quality and reliability of the software has become the major objective of software development industry. Researchers and industry practitioners, look for innovative techniques and methodologies that could be used to increase their confidence in the software reliability. Fault tree analysis (FTA) is one method under study at the Software Assurance Technology Center (SATC) of NASAs Goddard Space Flight Center to determine its relevance to increasing the quality and the reliability of software. This paper briefly reviews some of the previous research in the area of software fault tree analysis (SFTA). Next we discuss a roadmap for application of the SFTA to software, with special emphasis on object-oriented design. This is followed by a brief discussion of the paradigm for transforming a software design artifact (i.e., sequence diagram) to its corresponding software fault tree. Finally, we discuss challenges, advantages and disadvantages of SFTA.

Toward credible IT testing and certification

Accessible, rigorous measurement and test methods are key to creating quality software and increasing IT market competition. At the US National Institute of Standards and Technologys Information Technology Laboratory, work focuses on establishing comprehensive certification capability for the IT industry. This article describes NISTs work and how it can be applied.

Verification, validation & accreditation (panel): disciplines in dialogue or can we learn from the experiences of others?

Relative to Verification, Validation, and Accreditation (W&A) the Modeling and Simulation and Software Engineering communities share similar goals and impediments to achieving those goals. The focus of this panel is to explore how each community addresses the critical issues underlying VV&A. In this paper we provide nine (9) questions and four (4) sets of responses to those questions. The questions are intended to help reveal differences in VV&A emphasis and motivation between the two communities, and to establish a basis for the exchange of mutually beneficial ideas. Respondents

Center for High Integrity Software System Assurance

Abstract In order to enable the U.S. software industry to build high integrity software and to provide U.S. industries and government confidence in the software systems on which they are dependent, NIST created the Center for High Integrity Software System Assurance (CHISSA), which establishes criteria for software assurance for use by those who build or evaluate these systems. This paper describes the goals and organization of CHISSA.