Martin Weiglhofer

Specify, Compile, Run: Hardware from PSL

We propose to use a formal specification language as a high-level hardware description language. Formal languages allow for compact, unambiguous representations and yield designs that are correct by construction. The idea of automatic synthesis from specifications is old, but used to be completely impractical. Recently, great strides towards efficient synthesis from specifications have been made. In this paper we extend these recent methods to generate compact circuits and we show their practicality by synthesizing a generalized buffer and an arbiter for ARMs AMBA AHB bus from specifications given in PSL. These are the first industrial examples that have been synthesized automatically from their specifications.

Automatic Hardware Synthesis from Specifications: A Case Study

We propose to use a formal specification language as a high-level hardware description language. Formal languages allow for compact, unambiguous representations and yield designs that are correct by construction. The idea of automatic synthesis from specifications is old, but used to be completely impractical. Recently, great strides towards efficient synthesis from specifications have been made. In this paper we extend these recent methods to generate compact circuits and we show their practicality by synthesizing an arbiter for ARMs AMBA AHB bus and a generalized buffer from specifications given in PSL. These are the first industrial examples that have been synthesized automatically from their specifications

Protocol Conformance Testing a SIP Registrar: an Industrial Application of Formal Methods

Various research prototypes and a well-founded theory of model based testing (MBT) suggests the application of MBT to real-world problems. In this article we report on applying the well-known TGV tool for protocol conformance testing of a Session Initiation Protocol (SIP) server. Particularly, we discuss the performed abstractions along with corresponding rationales. Furthermore, we show how to use structural and fault-based techniques for test purpose design. We present first empirical results obtained from applying our test cases to a commercial implementation and to a popular open source implementation of a SIP Registrar. Notably, in both implementations our input output labeled transition system model proved successful in revealing severe violations of the protocol.

Automated Conformance Verification of Hybrid Systems

Due to the combination of discrete events and continuous behavior the validation of hybrid systems is a challenging task. Nevertheless, as for other systems the correctness of such hybrid systems is a major concern. In this paper we present a new approach for verifying the input-output conformance of two hybrid systems. This approach can be used to generate mutation-based test cases. We specify a hybrid system within the framework of Qualitative Action Systems. Here, besides conventional discrete actions, the continuous dynamics of hybrid systems is described with so called qualitative actions. This paper then shows how labeled transition systems can be used to describe the trace semantics of Qualitative Action Systems. The labeled transition systems are used to verify the conformance between two Qualitative Action Systems. Finally, we present first experimental results on a water tank system.

Asynchronous Input-Output Conformance Testing

This paper studies model-based input-output conformance testing in the presence of queues. Normally, it is assumed that a test case communicates synchronously with an implementation under test. This causes some challenges in practice, since testing is often conducted asynchronously. In an asynchronous environment messages between a tester and the implementation are queued. This may lead to incorrect verdicts. In this paper we show how one can guarantee correct verdicts in the asynchronous case for a large set of implementations. If choices between inputs and outputs are restricted to internal choices with respect to an implementation one can use the observation of quiescence as a handshake between a test case and the implementation. Such a handshake allows us to test for input-output conformance in the context of queues. In addition, the input-enabledness assumption on implementations is relaxed. Besides a formal discussion of this approach, we show the practical relevance by applying our approach to a conference protocol.

A Teleo-Reactive Architecture for Fast, Reactive and Robust Control of Mobile Robots

One of the elementary tasks of an autonomous mobile robot is the execution of different behavior patterns in order to fulfill a given task. The complexity of this problem is especially high if the robot operates in a dynamic, unpredictable environment and requires the parallel control of multiple actuators. In this paper we present a novel architecture for robust and fast mobile robot control. The architecture is based on Teleo-Reactive Programs. We discuss the benefits and drawbacks of such programs, extend the basic definition for the parallel control of multiple actuators, and propose a new language and a compiler for extended Teleo-Reactive Programs. These tools simplify the creation of new behavior patterns and increase the runtime performance. Finally, we discuss implementation issues of the architecture when applying it to RoboCup Middle-Size soccer robots.

When BDDs Fail: Conformance Testing with Symbolic Execution and SMT Solving

Model-based testing is a well known technique that allows one to validate the correctness of software with respect to its model. If a lot of data is involved, symbolic techniques usually outperform explicit data enumeration. In this paper, we focus on a new symbolic test case generation technique. Our approach is based on symbolic execution and on satisfiability (modulo theory; SMT) solving. Our work was motivated by the complete failure of a well-known existing symbolic test case generator to produce any test cases for an industrial Session Initiation Protocol (SIP) implementation. Hence, we have replaced the BDD-based analysis of the existing tool with a combination of symbolic execution and SMT solving. Our new tool generates the test cases for SIP in seconds. However, further experiments showed that our approach is not a substitutive but a complementary approach: we present the technique and the results obtained for two protocol specifications, the first supporting our new technique, the second being witness for the classic BDD-technique.

Improving Fault-based Conformance Testing

Fault-based conformance testing is a conformance testing strategy that relies on specific fault models. Previously, this mutation testing technique has been applied to protocol specifications. Although a practical case study of web-server testing has been conducted, we observed several issues when applying this method in a large industrial project. In this paper, we discuss the foundations, techniques and tools to overcome these shortcomings. More specifically, we show a solution to the problem of state-space explosion in generating mutation tests for industrial scale applications. Furthermore, the previous approach used the counterexamples of a bisimulation check (between the original and the mutant) as test purposes. With respect to input-output conformance (ioco), this is an over-approximation resulting in more tests than are necessary. Hence, we propose to use an ioco-checker in order to generate less test cases. An industrial case study demonstrates these improvements.

Using coverage to automate and improve test purpose based testing

Test purposes have been presented as a solution to avoid the state space explosion when selecting test cases from formal models. Although such techniques work very well with regard to the speed of the test derivation, they leave the tester with one important task that influences the quality of the overall testing process: test purposes have to be formulated manually. In this paper, we present an approach that assists a test engineer with test purpose design in two ways: it allows automatic generation of coverage based test suites and can be used to automatically exercise those aspects of the system that are missed by hand-crafted test purposes. We consider coverage of Lotos specifications, and show how labeled transition systems derived from such specifications have to be extended in order to allow the application of logical coverage criteria to Lotos specifications. We then show how existing tools can be used to efficiently derive test cases and suggest how to use the coverage information to minimize test suites while generating them.

Coverage Based Testing with Test Purposes

Test purposes have been presented as a solution to avoid the state space explosion when selecting test cases from formal models. Although such techniques work very well with regard to the speed of the test derivation, they leave the tester with one important task that influences the quality of the overall testing process: Test purposes have to be formulated manually. In this paper, we present an approach that assists a test engineer with test purpose design in two ways: It allows automatic generation of coverage based test suites and can be used to automatically exercise those aspects of the system that manually test purposes missed. We consider coverage of LOTOS specifications, and show how labeled transition systems derived from such specifications have to be extended in order to allow the application of logical coverage criteria to LOTOS specifications. We then show how existing tools can be used to efficiently derive test cases and suggest how to use the coverage information to minimize test suites while generating them.