John D. Musa

Operational profiles in software-reliability engineering

A systematic approach to organizing the process of determining the operational profile for guiding software development is presented. The operational profile is a quantitative characterization of how a system will be used that shows how to increase productivity and reliability and speed development by allocating development resources to function on the basis of use. Using an operational profile to guide testing ensures that if testing is terminated and the software is shipped because of schedule constraints, the most-used operations will have received the most testing and the reliability level will be the maximum that is practically achievable for the given test time. For guiding regression testing, it efficiently allocates test cases in accordance with use, so the faults most likely to be found, of those introduced by changes, are the ones that have the most effect on reliability.<<ETX>>

Software reliability measurement

The quantification of software reliability is needed for the system engineering of products involving computer programs and the scheduling and monitoring of software development. It is also valuable for the comparative evaluation of the effectiveness of various design, coding, testing, and documentation techniques. This paper outlines a theory of software reliability based on execution or cpu time, and a concomitant model of the testing and debugging process that permits execution time to be related to calendar time. The estimation of parameters of the model is discussed. Application of the theory in scheduling and monitoring software projects is described, and data taken from several actual projects are presented.

A theory of software reliability and its application

The theory permits the estimation, in advance of a project, of the amount of testing in terms of execution time required to achieve a specified reliability goal [stated as a mean time to failure (MTTF)]. Execution time can then be related to calendar time, permitting a schedule to be developed. Estimates of execution time and calendar time remaining until the reliability goal is attained can be continually remade as testing proceeds, based only on the length of the execution time intervals between failures. The current MTTF and the number of errors remaining can also be estimated. Maximum likelihood estimation is employed, and confidence intervals are also established. The foregoing information is obviously very valuable in scheduling and monitoring the progress of program testing. A program has been implemented to compute the foregoing quantities. The reliability model that has been developed can be used in making system tradeoffs involving software or software and hardware components. It also provides a soundly based unit of measure for the comparative evaluation of various programming techniques that are expected to enhance reliability. The model has been applied to four medium-sized software development projects, all of which have completed their life cycles.

Validity of Execution-Time Theory of Software Reliability

This paper investigates the validity of the execution-time theory of software reliability. The theory is outlined, along with appropriate background, definitions, assumptions, and mathematical relationships. Both the execution time and calendar time components are described. The important assumptions are discussed. Actual data are used to test the validity of most of the assumptions. Model and actual behavior are compared. The development projects and operational computation center software from which the data have been obtained are characterized to give the reader some basis for judging the breadth of applicability of the concepts.

Software reliability-engineered testing

Software testing often results in delays to market and high cost without assuring product reliability. Software reliability engineered testing (SRET), an ATT it in no way implies that SRET is limited to telecommunications systems. SRET is based on the ATT only five of 30 proposed best current practices were approved in 1991.

The future generation of software: a management perspective

The engineering process that underlies software development is examined. A brief summary of how information technology has affected both institutions and individuals in the past few decades is given. Engineering with models and metrics is then discussed. Improving software quality, making software engineering technology more transferable, and transferring software technology into an organization are addressed. >

Quantifying software validation: when to stop testing?

The authors explain how to use reliability models to determine how much system testing to do and how to allocate resources among the components to be tested. They begin by discussing the basic concepts of software reliability. They examine the functions, presuppositions, and basic procedures of system testing, as well as testing comparison and the use of calendar-time component to predict when testing will be completed. They then discuss acceptance testing. The authors conclude with examples of actual applications.<<ETX>>

The operational profile

Operational profiles are an important part of the technology and practice of software reliability engineering. The concept was developed originally (Musa et al. 1987) to make it possible to specify the nature of the use of a software-based system so that testing could be made as realistic as possible and so that reliability measurements would reflect that realism. However, the operational profile rapidly became useful for additional purposes in software reliability engineering (Musa 1993). In fact, it is also proving useful for purposes outside of software reliability engineering as well. This paper gives an overview of operational profile practice, discussing what the operational profile is, why it is important, and how it is developed and applied. It also presents some current open research questions; work in these areas can be expected to affect the practice of the future.

Criteria for Software Reliability Model Comparisons

A set of criteria is proposed for the comparison of software reliability models. The intention is to provide a logically organized basis for determining the superior models and for the presentation of model characteristics. It is hoped that in the future, a software manager will be able to more easily select the model most suitable for his/her requirements from among the preferred ones.

The operational profile in software reliability engineering: an overview

Provides an overview of a step-by-step approach for developing the operational profile used in software reliability engineering. The operational profile can help guide managerial and engineering decisions throughout the product life cycle, including requirements specification, design, implementation and testing. It is particularly valuable in guiding test planning and testing. The development and use of this profile typically involves product and market managers and systems engineers. It is used by software developers and managers, system test planners and testers, and quality assurance and reliability engineers. The presented material has been drawn from experience in developing operational profiles for a variety of applications.<<ETX>>