Michael Dyer

A formal approach to software error removal

Abstract There is a dichotomy of opinion on the use of software testing versus formal verification in software development. Testing has been the accepted method for detecting and removing errors and has played a significant error removal role. Formal verification has only recently matured into accepted practice but shows the potential for playing an even more significant error prevention role. The Cleanroom software development process which has been developed by the IBM Federal Systems Division combines both ideas into an effective development tool. Software engineering methods based on functional verification support the production of software with sufficient quality to forego traditional unit or structural testing. Statistical methods are introduced that define objective and formal strategies for product or functional testing. The synergism between the two ideas results in software with fewer errors which are both easier to find and to fix and in products with exceptional operating characteristics. Error prevention, not removal, is the key and the only viable approach to any sustained software quality growth. The Cleanroom development method and its impact on the error prevention and removal processes are covered in this paper. The results from its use for software development are also discussed.

Verification based inspection

A realization from cleanroom experience is that the software inspection can be driven by correctness verification ideas. Specifically, the questions to be asked by inspectors and the order in which they are to be asked can be automatically created from correctness proofs. The verification based inspection is qualitatively superior to the formal inspection in developing software with near zero defects.<<ETX>>

The management of software engineering: part iv: software development practices

Software engineering may be defined as the systematic design and development of software products and the management of the software process. Software engineering has as one of its primary objectives the production of programs that meet specifications, and are demonstrably accurate, produced on time, and within budget. This paper in five parts discusses the principles and practices used by the IBM Federal Systems Division for the design, development, and management of software. Part IV, on software engineering development practices, discusses a methodology for translating designs into software products. The subject is treated under two main headings, code management and integration engineering. These are rigorous methods for building the parts and integrating them into the whole software product that meets the design specifications.

Correctness verification: Alternative to structural software testing

Abstract Recent experience with formal software design methods has shown that software of significant quality can be produced without a programmer performing traditional structural testing of the code. This process improvement comes from the use of correctness verification methods in the development and subsequent inspection of the software design. While improved software quality could be anticipated from the use of verification methods, the accompanying productivity gains were not. This unexpected productivity prompted the study of the interaction between structural software testing and the verification methods. The outlook is that software of significant size and complexity can be developed without traditional structural testing by the programmer and with earlier and more sustained error prevention.

An approach to software reliability measurement

Abstract This paper discusses an approach to the definition and measurement of software reliability which is embodied in statistical testing concept. Differences in the software and hardware reliability approaches are covered and the statistical modelling techniques for the software case are described. Statistical methods used in the random sampling of test inputs and the validation of model predictions are reviewed. Data from project experments is included to illustrate the application of the software reliability concepts in actual product developments.

Designing software for provable correctness

Abstract Formal software design methods significantly improve the quality of software designs and products. They introduce new levels of defect prevention with a rigorous design process for decomposing product requirements, systematically verifying the correctness of each decomposition and maintaining requirements integrity throughout the process. The process has been effectively applied to software of significant size and complexity and has the potential for software development without the traditional debugging step. Methodology for software design is important since the majority of software defects (40%–60%) are introduced during the design step and the cost of their removal can be 100 times more expensive than defects introduced at later steps in the development process. The formal methods result in significantly fewer total defects than is currently realized (in the 0–20 defect range) and in their earlier detection and removal (90% prior to code execution).

Distribution-based statistical sampling: an approach to software functional test

Abstract Software testing uses both structural and functional strategies to ensure the delivery of quality products. Structural testing checks the implementation of the software design, whereas functional testing validates the correct implementation of the specified software requirements. Exhaustive functional testing for software of any reasonable complexity is recognized as being impractical; therefore, sampling strategies must be introduced to prioritize and optimize the requirements validation. Recent work with statistically based sampling techniques shows a potential for the systematic validation of requirements with the use of representative inputs from the softwares planned operating environments. This article discusses the need for sampling strategies in functional testing, the characteristics of and experience with a statistical sampling approach, and the impact of a statistical approach on the functional test process.

Integrating an enterprise's engineering processes

Abstract A case study is reported on the integration of engineering practices which had been separately prepared into a seamless system development process. The approach taken in the study should be usable by any enterprise which is attempting to effect a closer integration of the engineering skills in its systems developments. In the particular case, the engineering practices had been defined and institutionalized over the past several decades by the individual engineering disciplines to define their roles and responsibilities within the development life-cycle. At a macro process level, there was apparent integration between these practices but, as the study revealed, additional definition was generally required to ensure integration at the working level. A reasonably seamless development process to the working level was defined and documented through a combination of work flows and tabular descriptions of engineering activities and work products. The roles of multidisciplined and/or concurrent engineering teams in accomplishing key parts of the system development were emphasized to ensure close skill integration at the working level. The opportunities for collecting measurements on the integrated effort and for considering the reuse of process materials were also stressed in the process elaboration. Further integration of additional project disciplines within this integrated process is an ongoing research effort.