Marius Mikučionis

Testing real-time systems using UPPAAL

This chapter presents principles and techniques for modelbased black-box conformance testing of real-time systems using the Uppaal model-checking tool-suite. The basis for testing is given as a network of concurrent timed automata specified by the test engineer. Relativized input/output conformance serves as the notion of implementation correctness, essentially timed trace inclusion taking environment assumptions into account. Test cases can be generated offline and later executed, or they can be generated and executed online. For both approaches this chapter discusses how to specify test objectives, derive test sequences, apply these to the system under test, and assign a verdict.

Testing real-time embedded software using UPPAAL-TRON: an industrial case study

UPPAAL-TRON is a new tool for model based online black-box conformance testing of real-time embedded systems specified as timed automata. In this paper we present our experiences in applying our tool and technique on an industrial case study. We conclude that the tool and technique is applicable to practical systems, and that it has promising error detection potential and execution performance.

Formal Methods and Testing

Model Based Testing with Labelled Transition Systems.- Model-Based Testing of Object-Oriented Reactive Systems with Spec Explorer.- Testing Real-Time Systems Using UPPAAL.- Coverage Criteria for State Based Specifications.- Testing in the Distributed Test Architecture.- Testing from X-Machine Specifications.- Testing Data Types Implementations from Algebraic Specifications.- From MC/DC to RC/DC: Formalization and Analysis of Control-Flow Testing Criteria.- Comparing the Effectiveness of Testing Techniques.- The Test Technology TTCN-3.- Testability Transformation - Program Transformation to Improve Testability.- Modelling the Effects of Combining Diverse Software Fault Detection Techniques.

Time for statistical model checking of real-time systems

We propose the first tool for solving complex (some undecidable) problems of timed systems by using Statistical Model Checking (SMC). The tool monitors several runs of the system, and then relies on statistical algorithms to get an estimate of the correctness of the entire design. Contrary to other existing toolsets, ours relies on i) a natural stochastic semantics for networks of timed systems, ii) an engine capable to solve problems that are beyond the scope of classical model checkers, and iii) a friendly user interface.

Statistical model checking for networks of priced timed automata

This paper offers a natural stochastic semantics of Networks of Priced Timed Automata (NPTA) based on races between components. The semantics provides the basis for satisfaction of probabilistic Weighted CTL properties (PWCTL), conservatively extending the classical satisfaction of timed automata with respect to TCTL. In particular the extension allows for hard real-time properties of timed automata expressible in TCTL to be refined by performance properties, e.g. in terms of probabilistic guarantees of time- and cost-bounded properties. A second contribution of the paper is the application of Statistical Model Checking (SMC) to efficiently estimate the correctness of non-nested PWCTL model checking problems with a desired level of confidence, based on a number of independent runs of the NPTA. In addition to applying classical SMC algorithms, we also offer an extension that allows to efficiently compare performance properties of NPTAs in a parametric setting. The third contribution is an efficient tool implementation of our result and applications to several case studies.

UPPAAL-SMC: Statistical Model Checking for Priced Timed Automata ∗

This paper offers a survey of UPPAAL-SMC, a major extension of the real-time verification tool UPPAAL. UPPAAL-SMC allows for the efficient analysis of performance properties of networks of priced timed automata under a natural stochastic semantics. In particular, U PPAAL-SMC relies on a series of extensions of the statistical model checking app roach generalized to handle real-time systems and estimate undecidable problems. UPPAAL-SMC comes together with a friendly user interface that allows a user to specify complex problems in an efficient manner as well as to get feedback in the form of probability distributions and compare probabilities to analyze performance aspects of systems. The focus of the survey is on the evolution of the tool ‐ including modeling and specification formalisms as well as techniques applied ‐ tog ether with applications of the tool to case studies.

Online testing of real-time systems using UPPAAL

We present the development of T-UPPAAL — a new tool for online black-box testing of real-time embedded systems from non-deterministic timed automata specifica- tions. It is based on a sound and complete randomized online testing algorithm and is im- plemented using symbolic state representation and manipulation techniques. We propose the notion of relativized timed input/output conformance as the formal implementation relation. A novelty of this relation and our testing algorithm is that they explicitly take environment assumptions into account, generate, execute and verify the result online using the UPPAAL on-the-fly model-checking tool engine. This paper introduces the principles behind the tool, describes the present implementation status, and future work directions.

Statistical Model Checking for Stochastic Hybrid Systems

This paper presents novel extensions and applications of the UPPAAL-SMC model checker. The extensions allow for statistical model checking of stochastic hybrid systems. We show how our race-based stochastic semantics extends to networks of hybrid systems, and indicate the integration technique applied for implementing this semantics in the UPPAAL-SMC simulation engine. We report on two applications of the resulting tool-set coming fr om systems biology and energy aware buildings.

Schedulability of herschel-planck revisited using statistical model checking

Schedulability analysis is a main concern for several embedded applications due to their safety-critical nature. The classical method of response time analysis provides an efficient technique used in industrial practice. However, the method is based on conservative assumptions related to execution and blocking times of tasks. Consequently, the method may falsely declare deadline violations that will never occur during execution. This paper is a continuation of previous work of the authors in applying extended timed automata model checking (using the tool UPPAAL) to obtain more exact schedulability analysis, here in the presence of non-deterministic computation times of tasks given by intervals [BCET,WCET]. Considering computation intervals makes the schedulability of the resulting task model undecidable. Our contribution is to propose a combination of model checking techniques to obtain some guarantee on the (un)schedulability of the model even in the presence of undecidability. Two methods are considered: symbolic model checking and statistical model checking. Symbolic model checking allows to conclude schedulability --- i.e. absence of deadline violations --- for varying sizes of BCET. However, the symbolic model checking technique is over-approximating for the considered task model and can therefore not be used for disproving schedulability. As a remedy, we show how statistical model checking may be used to generate concrete counter examples witnessing non-schedulability. In addition, we apply statistical model checking to obtain more informative performance analysis --- e.g. expected response times --- when the system is schedulable. The methods are demonstrated on a complex satellite software system yielding new insights useful for the company.

An Evaluation Framework for Energy Aware Buildings using Statistical Model Checking

Cyber-physical systems are to be found in numerous applications throughout society. The principal barrier to develop trustworthy cyber-physical systems is the lack of expressive modelling and specification formalisms supported by efficient tools and methodologies. To overcome this barrier, we extend in this paper the modelling formalism of the tool UPPAAL-SMC to stochastic hybrid automata, thus providing the expressive power required for modelling complex cyber-physical systems. The application of Statistical Model Checking provides a highly scalable technique for analyzing performance properties of this formalisms.A particular kind of cyber-physical systems are Smart Grids which together with Intelligent, Energy Aware Buildings will play a major role in achieving an energy efficient society of the future. In this paper we present a framework in UPPAAL-SMC for energy aware buildings allowing to evaluate the performance of proposed control strategies in terms of their induced comfort and energy profiles under varying environmental settings (e.g. weather, user behavior etc.). To demonstrate the intended use and usefulness of our framework, we present an application to the Hybrid Systems Verification Benchmark.