Sanjai Rayadurgam

Coverage based test-case generation using model checkers

Presents a method for automatically generating test cases according to structural coverage criteria. We show how a model checker can be used to automatically generate complete test sequences that provide a pre-defined coverage of any software development artifact that can be represented as a finite state model. Our goal is to help reduce the high cost of developing test cases for safety-critical software applications that require a certain level of coverage for certification, e.g. safety-critical avionics systems that need to demonstrate MC/DC (modified condition and decision) coverage of the code. We define a formal framework which is suitable for modeling software artifacts like requirements models, software specifications or implementations. We then show how various structural coverage criteria can be formalized and used to make a model checker provide test sequences to achieve this coverage. To illustrate our approach, we demonstrate how a model checker can be used to generate test sequences for MC/DC coverage of a small case example.

Auto-generating test sequences using model checkers: A case study

Use of model-checking approaches for test generation from requirement models have been proposed by several researchers. These approaches leverage the witness (or counter-example) generation capability of model-checkers for constructing test cases. Test criteria are expressed as temporal properties. Witness traces generated for these properties are instantiated to create complete test sequences, satisfying the criteria. State-space explosion can, however, adversely impact model-checking and hence such test generation. Thus, there is a need to validate these approaches against realistic industrial sized system models to learn how well these approaches scale. To this end, we conducted a case study using six models of progressively increasing complexity of the mode-logic in a flight-guidance system, written in the RSML− e language. We developed a framework for specification-based test generation using the NuSMV model-checker and code based test case generation using Java Pathfinder, and collected time and resource usage data for generating test cases using symbolic, bounded, and explicit state model-checking algorithms. This paper briefly discusses the approach, presents the results from the study and analyzes its implications.

Generating MC/DC adequate test sequences through model checking

We present a method for automatically generating test sequences to satisfy MC/DC like structural coverage criteria of software behavioral models specified in state-based formalisms. The use of temporal logic for characterizing test criteria and the application of model-checking techniques for generating test sequences to those criteria have been of interest in software verification research for some time. Nevertheless, criteria for which constraints span more than one test sequence, such as the modified condition/decision coverage (MC/DC) mandated for critical avionics software, cannot be characterized in terms of a single temporal property. This paper discusses a method for recasting two-sequence constraints in the original model as a single sequence constraint expressed in temporal logic on a slightly modified model. The test-sequence generated by a model-checker for the modified model can be easily separated into two different test-sequences for the original model, satisfying the given test criteria. The approach has been successful in generating MC/DC test sequences from a model of the mode-logic in a flight-guidance system.

Test-sequence generation from formal requirement models

This paper discusses a method for generating test sequences from state-based specifications. We show how a model checker can be used to automatically generate complete test sequences that will provide arbitrary structural coverage of requirements specified in a high-level language like SCR or RSML/sup -e/. We have defined a language independent formal foundation for test sequence generation using model checkers that is suitable for representing software artifacts like requirements models, software specifications, and code. This paper shows a concrete application of our formal framework for test generation in the requirements modeling domain. The framework allows one to define structural coverage criteria in terms of the formal model of a software artifact and describes how test sequences can be generated to satisfy those coverage criteria using a model-checker. The approach is illustrated using examples. We define various criteria in terms of the specification language, translate those into criteria in the formal framework, and demonstrate how we generate the test sequences.

Automatic abstraction for model checking software systems with interrelated numeric constraints

Model checking techniques have not been effective in important classes of software systems characterized by large (or infinite) input domains with interrelated linear and non-linear constraints over the input variables. Various model abstraction techniques have been proposed to address this problem. In this paper, we wish to propose domain abstraction based on data equivalence and trajectory reduction as an alternative and complement to other abstraction techniques. Our technique applies the abstraction to the input domain (environment) instead of the model and is applicable to constraint-free and deterministic constrained data transition system. Our technique is automatable with some minor restrictions.

Assurance-based Y2K testing

Describes assurance techniques for Year-2000 (Y2K) testing. The Y2K problem is an important issue in the computer industry today, and testing is still the main technique for quality assurance. There is a need to ensure that the software is reasonably safe from Y2K faults after testing. This paper uses a statistical model for ensuring this, and it explicitly models Y2K faults as well as the ripples induced by Y2K modifications. The paper then describes two processes that use the model in practice: a bottom-up process that can be used together with software development, and a top-down process that can be used when the project is almost completed. These processes can be easily embedded in an existing testing process with minimal changes and minimal extra effort.

Modeling and requirements on the physical side of cyber-physical systems

In a cyber-physical system (a system where the physical world interacts extensively with-often networked-software), the physical portion of the system resides in the continuous and continual domain. Thus, on the physical side of cyber-physical systems we will have to contend with not only real time requirements but also the continuous and continual nature of the system. This poses a new set of challenges for requirements engineering; we must write well defined requirements to address crucial issues not commonly addressed in the software domain. For example, the rate of change of a controlled variable, the time it takes for a controlled variable to settle sufficiently close to a set-point, and the cumulative errors built up over time may be of critical importance. In this paper we outline how early modeling in the continuous domain serves as a crucial aid in the elicitation and discovery of requirements for cyber-physical systems and provide an initial classification of the types of requirements needed to describe crucial aspects of the physical side of a cyber-physical system.

Automated test-data generation from formal models of software

Verification and Validation (V&V) of software for critical embedded control systems often consumes upto 70% of the development resources. Testing is one of the most frequently used V&V technique for verifying such systems. Many regulatory agencies that certify control systems for use require that the software be tested to certain specified levels of coverage. Currently, developing test cases to meet these requirements takes a major portion of the resources. Automating this task would result in significant time and cost savings. The objective of this paper is to automate the generation of such test cases. We propose an approach where we rely on a formal model of the required software behavior for test-case generation, as well as, an oracle to determine if the implementation produced the correct output during testing.

Revenue-based call admission control for wireless cellular networks

Call admission control (CAC) schemes in wireless cellular networks attempt to reduce call dropping probability possibly at the expense of increased call blocking probability. We propose using channel reassignments in a controlled manner to minimize call dropping while maintaining high spectrum utilization. Guard channels are used to control the number of reassignments. The number of guard channels is dynamically determined using reassignment frequency as feedback. A simple scheme that attempts to maintain the number of reassignments under a specified target is described. A revenue-based CAC scheme is then presented which attempts to maximize income by balancing the penalty for reassignments against the reward for serviced calls. Simulation results confirm and validate the ideas discussed.

Experience in capturing requirements for safety-critical medical devices in an industrial environment

This paper presents some of the lessons learned in developing safety-critical implantable medical devices, such as pacemakers and defibrillators, in an industrial environment. It discusses some important issues related to obtaining requirements directly from end users, and their impact on reliability and safety aspects of the system. The emphasis is on practical aspects of system and software development rather than on theoretical aspects.