Jesse H. Poore

Planning and certifying software system reliability

An approach to software reliability and certification is presented that is based on the use of three mathematical models: the sampling, component, and certification models. The approach helps reduce reliability analysis to a problem that can be evaluated and manipulated through a series of spreadsheets. This approach was motivated by interest in applying the cleanroom software-engineering method in environments that require extensive code reuse.<<ETX>>

Foundations of sequence-based software specification

Rigorous specification early in the software development process can greatly reduce the cost of later development and maintenance, as well as provide an explicit means to manage risk and identify and meet safety requirements. Sequence-based software specification is a collection of techniques for implementing rigorous, practical software specification. The primary result of this research is the sequence enumeration method of specification writing. Straightforward, systematic enumeration of all sequences to produce an arguably complete, consistent, and traceably correct specification is made practical by controlling the growth of the process.

Generating transition probabilities to support model-based software testing

Markov chain usage models support test planning, test automation, and analysis of test results. In practice, transition probabilities for Markov chain usage models are often specified using a cycle of assigning, verifying, and revising specific values for individual transition probabilities. For large systems, such an approach can be difficult for a variety of reasons. We describe an improved approach that represents transition probabilities by explicitly preserving the information concerning test objectives and the relationships between transition probabilities in a format that is easy to maintain and easy to analyze. Using mathematical programming, transition probabilities are automatically generated to satisfy test management objectives and constraints. A more mathematical treatment of this approach is given in References [1] (Poore JH, Walton GH, Whittaker JA. A constraint‐based approach to the representation of software usage models. Information and SoftwareTechnology 2000; at press) and [2] (Walton GH. Generating transition probabilities for Markov chain usage models. PhD Thesis, University of Tennessee, Knoxville, TN, May 1995.). In contrast, this paper is targeted at the software engineering practitioner, software development manager, and test manager. This paper also adds to the published literature on Markov chain usage modeling and model‐based testing by describing and illustrating an iterative process for usage model development and optimization and by providing some recommendations for embedding model‐based testing activities within an incremental development process. Copyright

Computing system reliability using Markov chain usage models

Markov chains have been used successfully to model system use, generate tests, and compute statistics about anticipated system use in the field. Several reliability models are in use for Markov chain-based testing, but each has certain limitations. A Bayesian reliability model that is gaining support in field use is presented here.

A constraint-based approach to the representation of software usage models

Abstract Software usage models are the basis for statistical testing. They derive their structure from specifications and their probabilities from evolving knowledge about the intended use of the software product. The evolving knowledge comes from developers, customers and testers of the software system in the form of relationships that should hold among the parameters of a model. When software usage models are encoded as Markov chains, their structure can be represented by a system of linear constraints, and many of the evolving relationships among model parameters can be represented by convex constraints. Given a Markov chain usage model as a system of convex constraints, mathematical programming can be used to generate the Markov chain transition probabilities that represent a specific software usage model.

Statistical testing for cleanroom software engineering

Cleanroom software engineering requires statistical testing by an independent agent for the purpose of certifying software quality. Statistical software testing is a formal process that involves sampling from the intended usage environment and the precise measurement of properties of random variables inherent in such a statistical experiment. The first step in conducting a statistical test is to determine the usage distribution for the software in its intended environment. This distribution is the basis for a random generator of test sequences for the software. In order to achieve this distribution, a usage analysis is performed using the software specification and any available usage information. The usage analysis consists of a top down, structural investigation of the specification document that establishes a set of usage states and defines an ordering on this set. The paper describes in detail the usage analysis and inference procedure including various computations on the ensuing Markov chains. Stopping criteria are developed and a discrete software reliability model is presented.<<ETX>>

Sequence-based software specification of deterministic systems

Specification of software under the box structure method requires a complete, consistent, and traceably‐correct description of behavior solely in terms of external stimuli and responses. Such a specification, also called a black box, can be derived from the requirements through straightforward, systematic enumeration of all stimulus sequences. Enumeration is made manageable by the application of techniques for controlling the growth of this inherently combinatorial process, and specifications at different levels of abstraction may be combined to refine a black box specification. This work presents a unifying framework for development of specifications and testing models, and the focus on requirements traceability provides an explicit means to manage requirements change.

Stopping criteria for statistical testing

Abstract The decision to stop testing can be based on a number of criteria, such as (1) the confidence in a reliability estimate; (2) the degree to which testing experience has converged to the expected use of the software; and (3) model coverage criteria based on a degree of state, arc, or path coverage during crafted and random testing. In practice it is best to use multiple stopping criteria. For example, further evaluation of the testing performed is needed if the measure of correspondence between testing experience and expected use of the software indicates that the testing experience closely matches the expected use of the software, yet the variance of the reliability estimate is unacceptably large. One test of equality of testing experience and expected use is the Kullback discriminant from the usage chain to the testing chain. A new measure of “approximate equality” is introduced here for use in conjunction with the Kullback discriminant.

Partition testing with usage models

Abstract The fundamental statistical strategy of improving sampling efficiency through partitioning the population is applied to software testing. Usage models make it possible to apply this strategy to improve the efficiency of testing. The testing budget is allocated to the blocks of the partition, and the software is executed on the sample of uses drawn from each block or sub-population of potential uses. Usage models support many strategies for automated partitioning and generating test cases from the partitioned population. Two strategies are shown here with the efficiency gains demonstrated.

Sequence-based specification of feedback control systems in Simulink®

Solid-state microprocessors with software controlled sensors and actuators have essentially replaced analog control systems. Design systems with extensive libraries and code generators such as the ETAS® ASCET and MATLAB®/Simulink are widely used in industry to design control systems. However. the software engineering methods to help get the design right are missing. Sequence-based specification is a rigorous method that is well suited to the design of embedded control systems. This paper focuses on the adaptation of sequence-based specification to Simulink blocks, feedback control, and state machine generation, while preserving the ability to convert ordinary requirements to precise state-machine specifications.