Anders Hessel

Testing real-time systems using UPPAAL

This chapter presents principles and techniques for modelbased black-box conformance testing of real-time systems using the Uppaal model-checking tool-suite. The basis for testing is given as a network of concurrent timed automata specified by the test engineer. Relativized input/output conformance serves as the notion of implementation correctness, essentially timed trace inclusion taking environment assumptions into account. Test cases can be generated offline and later executed, or they can be generated and executed online. For both approaches this chapter discusses how to specify test objectives, derive test sequences, apply these to the system under test, and assign a verdict.

Formal Methods and Testing

Model Based Testing with Labelled Transition Systems.- Model-Based Testing of Object-Oriented Reactive Systems with Spec Explorer.- Testing Real-Time Systems Using UPPAAL.- Coverage Criteria for State Based Specifications.- Testing in the Distributed Test Architecture.- Testing from X-Machine Specifications.- Testing Data Types Implementations from Algebraic Specifications.- From MC/DC to RC/DC: Formalization and Analysis of Control-Flow Testing Criteria.- Comparing the Effectiveness of Testing Techniques.- The Test Technology TTCN-3.- Testability Transformation - Program Transformation to Improve Testability.- Modelling the Effects of Combining Diverse Software Fault Detection Techniques.

Time-Optimal Real-Time Test Case Generation Using Uppaal

Testing is the primary software validation technique used by industry today, but remains ad hoc, error prone, and very expensive. A promising improvement is to automatically generate test cases from formal models of the system under test.

Specifying and generating test cases using observer automata

We present a technique for specifying coverage criteria and a method for generating test suites for systems whose behaviours can be described as extended finite state machines (EFSM). To specify coverage criteria we use observer automata with parameters, which monitor and accept traces that cover a given test criterion of an EFSM. The flexibility of the technique is demonstrated by specifying a number of well-known coverage criteria based on control- and data-flow information using observer automata with parameters. We also develop a method for generating test cases from coverage criteria specified as observers. It is based on transforming a given observer automata into a bitvector analysis problem that can be efficiently implemented as an extension to an existing state-space exploration such as, e.g. SPIN or Uppaal.

A Global Algorithm for Model-Based Test Suite Generation

Abstract Model-based testing has been proposed as a technique to automatically verify that a system conforms to its specification. A popular approach is to use a model-checker to produce a set of test cases by formulating the test generation problem as a reachability problem. To guide the selection of test cases, a coverage criterion is often used. A coverage criterion can be seen as a set of items to be covered, called coverage items. We propose an on-the-fly algorithm that generates a test suite that covers all feasible coverage items. The algorithm returns a set of traces that includes a path fulfilling each item, without including redundant paths. The reachability algorithm explores a state only if it might increase the total coverage. The decision is global in the sense that it does not only regard each individual local search branch in isolation, but the total coverage in all branches together. For simpler coverage criteria as location of edge coverage, this implies that each model state is never explored twice. The algorithm presented in this paper has been implemented in the test generation tool Uppaal co ✓ er . We present encouraging results from applying the tool to a set of experiments and in an industrial sized case study.

Model-based testing of a WAP gateway: an industrial case-study

We present experiences from a case study where a model-based approach to black-box testing is applied to verify that aWireless Application Protocol (WAP) gateway conforms to its specification. TheWAP gateway is developed by Ericsson and used in mobile telephone networks to connect mobile phones with the Internet. We focus on testing the software implementing the session (WSP) and transaction (WTP) layers of the WAP protocol. These layers, and their surrounding environment, are described as a network of timed automata. To model the many sequence numbers (from a large domain) used in the protocol, we introduce an abstraction technique. We believe the suggested abstraction technique will prove useful to model and analyse other similar protocols with sequence numbers, in particular in the context of model-based testing. A complete test bed is presented, which includes generation and execution of test cases. It takes as input a model and a coverage criterion expressed as an observer, and returns a verdict for each test case. The test bed includes existing tools from Ericsson for test-case execution. To generate test suites, we use our own tool CO/ER--a new test-case generation tool based on the real-time model-checker UPPAAL.

Time-Optimal Test Cases for Real-Time Systems

Testing is the primary software validation technique used by industry today, but remains ad hoc, error prone, and very expensive. A promising improvement is to automatically generate test cases from formal models of the system under test.

Monitoring and Testing with Case Observer Automata: An Industry Report

In a highly configurable system, new or changed configurations may have to be tested in a running environment. To elicit high level information from such system, a monitoring solution must be able to: sort out the interesting data, connect the related data, draw conclusions about the data, and report the conclusion. The conclusions can be helpful to verify that the new configuration has the desired functionality. To continuously monitor requirements and try to find specific erroneous pattern is helpful during testing but when the system is deployed. In this industry report we present the Case Observer Automata modeling language (COAml) that aims to solve these issues. The COAml works with workflows where related data are put into cases, which can be seen as instances of the same observation pattern. In a case, parallel activities can be traced with synchronization points and timers. The Case Observer Language can be integrated into most forms of tracing/monitoring such as business rule monitoring, coverage tracing, static program analysis, log analysis, etc. We are currently integrating the language in the xTrade Alarm Server at Xware, and there are plans to reintegrate the COAml into UPPAAL Cover.