Robert Eschbach

Risk-Based Testing of Safety-Critical Embedded Systems Driven by Fault Tree Analysis

One important aspect of the quality assurance process of safety-critical embedded systems is verifying the appropriateness, correctness of the implementation and effectiveness of safety functions. Due to the rapid growth in complexity, manual verification activities are no longer feasible. This holds especially for testing. A popular method for testing such complex systems is model-based testing. Recent techniques for model-based testing do not sufficiently take into consideration the information derived from the safety analyses like Failure Mode and Effect Analysis and Fault Tree Analyses (FTA). In this paper, we describe an approach to use the results of FTA during the construction of test models, such that test cases can be derived, selected and prioritized according to the severity of the identified risks and the number of basic events that cause it. This approach is demonstrated on an example from the automation domain, namely a modular production system. We find that the method provides a significant increase in coverage of safety functions, compared to regular model based testing.

From Requirements to Statistical Testing of Embedded Systems

This paper presents the results of a research project where the combination of techniques of sequence-based requirements specification and model- based statistical testing has been applied to a real mirror control unit of a car door for reliability estimations. A complete chain from a requirements document to a statistical test report with a very high degree of automation is demonstrated. A practical solution for reliability analysis of embedded systems in a realistic industrial setup is proposed.

Reducing test effort: A systematic mapping study on existing approaches

Context: Quality assurance effort, especially testing effort, is often a major cost factor during software development, which sometimes consumes more than 50% of the overall development effort. Consequently, one major goal is often to reduce testing effort. Objective: The main goal of the systematic mapping study is the identification of existing approaches that are able to reduce testing effort. Therefore, an overview should be presented both for researchers and practitioners in order to identify, on the one hand, future research directions and, on the other hand, potential for improvements in practical environments. Method: Two researchers performed a systematic mapping study, focusing on four databases with an initial result set of 4020 articles. Results: In total, we selected and categorized 144 articles. Five different areas were identified that exploit different ways to reduce testing effort: approaches that predict defect-prone parts or defect content, automation, test input reduction approaches, quality assurance techniques applied before testing, and test strategy approaches. Conclusion: The results reflect an increased interest in this topic in recent years. A lot of different approaches have been developed, refined, and evaluated in different environments. The highest attention was found with respect to automation and prediction approaches. In addition, some input reduction approaches were found. However, in terms of combining early quality assurance activities with testing to reduce test effort, only a small number of approaches were found. Due to the continuous challenge of reducing test effort, future research in this area is expected.

On the Formal Semantics of SDL-2000: A Compilation Approach Based on an Abstract SDL Machine

In November 1999, a new version of SDL (Specification and Description Language) called SDL-2000 has passed ITU, an international standardization body for telecommunication. SDL is a fairly complex, graphical formal description technique for the development of distributed systems, and has been broadly used in industry for many years. Efforts to define the semantics of SDL-2000 formally have started early in 1998. By now, a draft formal semantics is available, which is determined to become the official formal SDL semantics after its approval in 2000. It is based on the formalism of Abstract State Machines (ASMs), which has been selected for several reasons including intelligibility and executability. The formal semantics of SDL addresses the static semantics, transformation rules, and the dynamic semantics. The approach taken to define the dynamic semantics is particularly interesting. Although basically being operational, it differs from existing approaches in several ways. In this paper, we address and highlight some of these differences, using a simplified specification language called SSL instead of SDL. In defining a formal dynamic semantics for SSL, we formally describe an abstract machine, a compilation function mapping SSL specifications to code of this machine, and an operational definition of the set of initial states, using ASM as the underlying formalism. Furthermore, we present in some detail the semantics of SSL procedure calls.

Iterative refinement of specification for component based embedded systems

The current practice of component based engineering raises concerns in industry when the specification of proprietary components suffers from inaccuracy and incompleteness. Engineers face difficulties in producing quality systems when they lack knowledge of the interoperability of components. In order to address this issue, we present a novel framework for iterative refinement of specification for component based systems. The novelty is the use of a preliminary behavioral model as a source for triggering refinement iterations. Moreover, it exploits rigorous formal techniques to achieve high-level system validation as an integral part of the refinement procedure. The framework has been evaluated on an automotive system in which the embedded software control units were developed by third-party vendors. The final results produced an improved formal system specification that identified several behaviors that were previously unknown.

A Systematic Approach to Construct Compositional Behaviour Models for Network-structured Safety-critical Systems

This paper considers the problem of model-based testing of a class of safety-critical systems. These systems are built up from components that are connected a network-like structure. The number of possible structures is usually large. In particular, we consider the following issue: For many of these systems, each instance needs its own set of models for testing. On the other hand, the instances that should be tested will have to be chosen so that the reliability statements are generally applicable. Thus, they must be chosen by a domain expert. The approach in this paper addresses both of these points. The structure of the instance of system under test is described using a domain-specific language, so that a domain expert can easily describe a system instance for testing. At the same time, the components and composition operators are formalized. Using a structure description written in the DSL, corresponding test models can be automatically generated, allowing for automated testing by the domain expert. We show some evidence about the feasibility of our approach and about the effort required for modelling an example, supporting our belief that our approach improves both on the efficiency and the expressivity of current compositional test model construction techniques.

Automated fault tree generation and risk-based testing of networked automation systems

In manufacturing automation domain safety and availability are the most important factors to ensure productivity. In modern software intensive networked automation systems it became quite hard to ensure which non-functional requirements are related to these factors as well as whether these are satisfied or not. This is due to the prevalence of manual efforts in several analyses phases where complexity of the system often makes it hard to obtain comprehensive overview and thus makes it difficult to ascertain the presence of certain undesired consequences. Since design, development and following verification and validation activities are largely dependent upon the result of the analyses the product is largely affected. To address these problems automated fault tree generation is presented in this paper. It uses distinct modeling artifacts and information to automatically compose formal models of the system. Embedding hardware and network failures it is then ascertained through model checking whether the system satisfies certain safety and availability properties or not. This information is used to compose the fault tree. Proposed approach will improve completeness and correctness in fault trees and will consequently help in improving the quality of the system. Furthermore, it is also shown how the artifacts of this analysis can be used to produce test goals and test cases to validate the software constituents of the system and assure traceability between testing activity and safety requirements.

Generating System Models for a Highly Configurable Train Control System Using a Domain-Specific Language: A Case Study

In this work, we present a results from case study ontesting a highly con¿gurable, safety-critical system from therailway domain using model-based risk-oriented testing. Inthe construction of the system and test models, we face thefollowing problems: (i) A domain expert will usually not beknowledgeable in the construction of system models, but hasvery detailed knowledge which con¿gurations of the systemwill be especially critical (e.g., prone to head-on collisions).Thus, a method for the construction of system and testmodels from domain-speci¿c descriptions is necessary. (ii)The system model shall be validatable against the system’srequirements. (iii) The veri¿cation of the system modelagainst safety requirements should be possible. We willdemonstrate an approach based on DSLs, compositionalconstruction of Mealy machines and a proof technique asa solution to these three problems.

A Termination Detection Algorithm: Specification and Verification

We propose a methodology for the specification and verification of distributed algorithms using Gurevichs concept of Abstract State Machines. The methodology relies on a distinction between a higher-level specification and a lower-level specification of an algorithm. The algorithm is characterized by an informal problem description. A justification assures the appropriateness of the higher-level specification for the problem description. A mathematical verification assures that the lower-level specification implements the higher-level one and is based on a refinement-relation. This methodology is demonstrated by a wellknown distributed termination detection algorithm originally invented by Dijkstra, Feijen, and van Gasteren.

SIMOTEST: A tool for automated testing of hybrid real-time Simulink models

This paper is presenting a tool called SIMOTEST. It allows the user to conduct Model Based Testing of MatLab/Simulink models. In particular it allows automatically running and evaluating test cases which got generated from a test model. Test cases which get run from SIMOTEST can be used for testing hybrid realtime systems i.e. systems which have continuous and discrete inputs/outputs which need to fulfill timing requirements. This is achieved by using the IEEE Standard for Signal and Test Definition (IEEE 1641).