Ingo Pill

Formal analysis of hardware requirements

Formal languages are increasingly used to describe the functional requirements (specifications) of circuits. These requirements are used as a means to communicate design intent and as basis for verification. In both settings it is of utmost importance that the specifications are of high quality. However, formal requirements are seldom the object of validation, even though they can be hard to understand and interactions between them can be subtle. In this paper, we present techniques and guidelines to explore and assure the quality of a formal specification. We define a technique to interactively explore the semantics of a specification by simulating its behavior for user-defined scenarios. Furthermore, we define techniques to automatically check specifications against a set of user-provided assertions, which must be satisfied, and a set of possibilities, which must not be contradicted. The proposed techniques support the user in the iterative development and refinement of high-quality specifications

Belief management for high-level robot programs

The robot programming and plan language IndiGolog allows for on-line execution of actions and offline projections of programs in dynamic and partly unknown environments. Basic assumptions are that the outcomes of primitive and sensing actions are correctly modeled, and that the agent is informed about all exogenous events beyond its control. In real-world applications, however, such assumptions do not hold. In fact, an actions outcome is error-prone and sensing results are noisy. In this paper, we present a belief management system in IndiGolog that is able to detect inconsistencies between a robots modeled belief and what happened in reality. The system furthermore derives explanations and maintains a consistent belief. Our main contributions are (1) a belief management system following a history-based diagnosis approach that allows an agent to actively cope with faulty actions and the occurrence of exogenous events; and (2) an implementation in IndiGolog and experimental results from a delivery domain.

Optimizations for the Boolean approach to computing minimal hitting sets

The Boolean approach to computing minimal hitting sets proposed by Lin and Jiang is known to offer very attractive general performance, but also has its issues, specifically with a cardinality-restricted search. In this paper we propose optimizations regarding the refinement rules, also offering a revised decision strategy as well as optimized termination criteria that exploit cardinality bounds. Our experiments including artificial and real-world samples for the bounded and unbounded case show the potential of our work, where we could achieve speed-ups of up to two orders of magnitude.

Functional SOA testing based on constraints

In the fierce competition on todays software market, Service-Oriented Architectures (SOAs) are an established design paradigm. Essential concepts like modularization, reuse, and the corresponding IP core business are inherently supported in the development and operation of SOAs that offer flexibility in many aspects and thus optimal conditions also for heterogeneous system developments. The intrinsics of large and complex SOA enterprises, however, require us to adopt and evolve our verification technology, in order to achieve expected software quality levels. In this paper, we contribute to this challenge by proposing a constraint based testing approach for SOAs. In our work, we augment a SOAs BPEL business model with pre- and postcondition contracts defining essential component traits, and derive a suite of feasible test cases to be executed after assessing its quality via corresponding coverage criteria. We illustrate our approachs viability via a running example as well as experimental results, and discuss current and envisioned automation levels in the context of a test and diagnosis workflow.

Fifty Shades of Grey in SOA Testing

Testing is undisputedly a fundamental verification principle in the software landscape. Todays products require us to effectively handle and test huge, complex systems and in this context to tackle challenging traits like heterogeneity, distribution and controllability to name just a few. The advent of Service-Oriented Architectures with their inherent technological features like dynamics and heterogeneity exacerbated faced challenges, requiring us to evolve our technology. The traditional view of white or black box testing, for example, does not accommodate the multitude of shades of grey one should be able to exploit effectively for system-wide tests. Today, while there are a multitude of approaches for testing single services, there is still few work on methodological system tests for SOAs. In this paper we propose a corresponding workflow for tackling SOA testing and diagnosis, discuss SOA test case generation in more detail, and report preliminary research in that direction.

SOA Grey Box Testing -- A Constraint-Based Approach

Service-Oriented Architectures (SOAs) offer attractive advantages in respect of reusability, interoperability and dynamics, and are nowadays widely accepted in industry. Achieving established software quality levels also with SOAs, while mandatory, is challenging, as, for instance, a SOAs dynamics and heterogeneity exacerbate verification issues like observability, controllability, and distribution. Regarding verification, we thus have to evolve available technology in order to enable the assessment of essential functional and non-functional system properties, including correctness, performance, stability, robustness and scalability. Adopting a model-based grey box testing approach that can exploit mixed description levels for individual (possibly 3rd party) services promises the required flexibility for successful development workflows. In this paper, we propose such a testing approach that, considering a SOA model, defines constraint satisfaction problems for the test case generation step. First empirical results for our approach are promising.

On classification and modeling issues in distributed model-based diagnosis

With model-based diagnosis, diagnoses for occurring faults can be directly computed from a given system model and actual observations about system behavior. Model-based diagnosis has been successfully accommodated to several purposes, including the diagnosis of space probes and configuration knowledge bases. Recent research includes extensions for distributed systems, motivated by the ever-growing system complexity and inherently distributed domains like service-oriented architectures. Previous work in this context lacks however a detailed analysis and classification approach that considers essential underlying issues like diagnosis architecture, utilized models, and abstract requirements that might stem from the application domain. In this paper, we will show an analysis of distributed system diagnosis and a characterization in the three dimensions mentioned.

SIMULTATE: A Toolset for Fault Injection and Mutation Testing of Simulink Models

The advantages of fault injection techniques and related methodologies like mutation testing have been gaining in attention also from industry, as is evident from the advent of standards like ISO 26262 that suggest to use corresponding approaches for verifying an automotive systems safety aspects. Aside a well-established theoretical background, the availability of tools is a key issue in order to leverage fault injection for the development of industrial, possibly safety-critical applications, e.g., in an automotive context. We propose the corresponding open source toolset SIMULTATE for injecting faults and performing mutation testing for Simulink models. For complementing the provided mutation/fault injection operators, it allows a user to define her own ones within Matlab and further provides a Python interface for easily deriving mutants where she can also focus the scope to desired model parts only. Controlling the activation of individual faults in a derived model, a designer can conveniently conduct mutation tests via a corresponding Python application.

SOA Testing via Random Paths in BPEL Models

Service oriented architectures (SOAs) allow for interesting and flexible software designs for complex problems. Such complex designs, however, require us to pay special attention to ensuring their quality. Extending earlier work on a testing and diagnosis concept for SOAs, we report in this paper on our first experience regarding an experimental setup with a test case generation algorithm based on random path selection, in contrast to our earlier work being based on considering all paths in a SOAs BPEL process description.

BPEL Integration Testing

Service-oriented architectures, and evolvements such as clouds, provide a promising infrastructure for future computing. They encapsulate an IP core’s functionality for easy access via well-defined business and web interfaces, and in turn allow us to flexibly realize complex software drawing on available expertise. In this paper, we take a look at some challenges we have to face during the task of testing such systems for verification purposes. In particular, we delve into the task of test suite generation, and compare the performance of two corresponding algorithms. In addition, we report on experiments for a collection of BPEL processes taken from the literature, in order to identify performance trends with respect to fault coverage metrics. Our results suggest that a structural reasoning might outperform a completely random approach.