Jochen Wuttke

Software Engineering for Self-Adaptive Systems: A Second Research Roadmap

The goal of this roadmap paper is to summarize the state-of-the-art and identify research challenges when developing, deploying and managing self-adaptive software systems. Instead of dealing with a wide range of topics associated with the field, we focus on four essential topics of self-adaptation: design space for self-adaptive solutions, software engineering processes for self-adaptive systems, from centralized to decentralized control, and practical run-time verification & validation for self-adaptive systems. For each topic, we present an overview, suggest future directions, and focus on selected challenges. This paper complements and extends a previous roadmap on software engineering for self-adaptive systems published in 2009 covering a different set of topics, and reflecting in part on the previous paper. This roadmap is one of the many results of the Dagstuhl Seminar 10431 on Software Engineering for Self-Adaptive Systems, which took place in October 2010.

On Patterns for Decentralized Control in Self-Adaptive Systems

Self-adaptation is typically realized using a control loop. One prominent approach for organizing a control loop in self-adaptive systems is by means of four components that are responsible for the primary functions of self-adaptation: Monitor, Analyze, Plan, and Execute, together forming a MAPE loop. When systems are large, complex, and heterogeneous, a single MAPE loop may not be sufficient for managing all adaptation in a system, so multiple MAPE loops may be introduced. In self-adaptive systems with multiple MAPE loops, decisions about how to decentralize each of the MAPE functions must be made. These decisions involve how and whether the corresponding functions from multiple loops are to be coordinated (e.g., planning components coordinating to prepare a plan for an adaptation). To foster comprehension of self-adaptive systems with multiple MAPE loops and support reuse of known solutions, it is crucial that we document common design approaches for engineers. As such systematic knowledge is currently lacking, it is timely to reflect on these systems to: (a) consolidate the knowledge in this area, and (b) to develop a systematic approach for describing different types of control in self-adaptive systems. We contribute with a simple notation for describing interacting MAPE loops, which we believe helps in achieving (b), and we use this notation to describe a number of existing patterns of interacting MAPE loops, to begin to fulfill (a). From our study, we outline numerous remaining research challenges in this area.

Model checking of UML 2.0 interactions

The UML 2.0 integrates a dialect of High-Level Message Sequence Charts (HMSCs) for interaction modelling. We describe a translation of UML 2.0 interactions into automata for model checking whether an interaction can be satisfied by a given set of message exchanging UML state machines. The translation supports basic interactions, state invariants, strict and weak sequencing, alternatives, ignores, and loops as well as forbidden interaction fragments. The translation is integrated into the UML model checking tool HUGO/RT.

Empirically revisiting the test independence assumption

In a test suite, all the test cases should be independent: no test should affect any other test’s result, and running the tests in any order should produce the same test results. Techniques such as test prioritization generally assume that the tests in a suite are independent. Test dependence is a little-studied phenomenon. This paper presents five results related to test dependence. First, we characterize the test dependence that arises in practice. We studied 96 real-world dependent tests from 5 issue tracking systems. Our study shows that test dependence can be hard for programmers to identify. It also shows that test dependence can cause non-trivial consequences, such as masking program faults and leading to spurious bug reports. Second, we formally define test dependence in terms of test suites as ordered sequences of tests along with explicit environments in which these tests are executed. We formulate the problem of detecting dependent tests and prove that a useful special case is NP-complete. Third, guided by the study of real-world dependent tests, we propose and compare four algorithms to detect dependent tests in a test suite. Fourth, we applied our dependent test detection algorithms to 4 real-world programs and found dependent tests in each human-written and automatically-generated test suite. Fifth, we empirically assessed the impact of dependent tests on five test prioritization techniques. Dependent tests affect the output of all five techniques; that is, the reordered suite fails even though the original suite did not.

Finding bugs by isolating unit tests

Even in simple programs there are hidden assumptions and dependencies between units that are not immediately visible in each involved unit. These dependencies are generally hard to identify and locate, and can lead to subtle faults that are often missed, even by extensive test suites. We propose to leverage existing test suites to identify faults due to hidden dependencies and to identify inadequate test suite design. Rather than just executing entire test suites within frameworks such as JUnit, we execute each test in isolation, thus removing masking effects that might be present in the test suites. We hypothesize that this can reveal previously hidden dependencies between program units or tests. A preliminary study shows that this technique is capable of identifying subtle faults that have lived in a system for 120 revisions, despite failures being reported and despite attempts to fix the fault.

Generating Effective Integration Test Cases from Unit Ones

Unit testing aims to ensure that methods correctly implement the specified and implied pre- and post-conditions, while integration testing ensures that modules correctly follow interaction protocols. While the generation of unit test cases has been explored extensively in the literature, there is still little work on the generation of integration test cases. In this paper we present a new technique to generate integration test cases that leverages existing unit test cases. Our key observation is that both, unit and integration testing, use method calls as the atoms to construct test cases from. Unit tests contain information on how to instantiate classes in meaningful ways, how to construct arguments for method calls, and what the resulting system state should be after calling methods with those arguments. We use this information to construct more complex test cases that focus on class interactions rather than on individual state transformations caused by single method calls. This paper presents the approach and shows that the generated test cases can find interesting faults, compared to test suites generated with state of the art approaches.

Traffic routing for evaluating self-adaptation

Toward improving the ability to evaluate self-adaptation mechanisms, we present the automated traffic routing problem. This problem involves large numbers of agents, partial knowledge, and uncertainty, making it well-suited to be solved using many, distinct self-adaptation mechanisms. The well-defined nature of the problem allows for comparison and proper evaluation of the underlying mechanisms and the involved algorithms. We (1) define the problem, (2) outline the sources of uncertainty and partial information that can be addressed by self-adaptation, (3) enumerate the dimensions along which self-adaptive systems should be evaluated to provide a benchmark for comparison of self-adaptation and traditional mechanisms, (4) present Adasim, an open-source traffic routing simulator that allows easy implementation and comparison of systems solving the automated traffic routing problem, and (5) demonstrate Adasim by implementing two traffic routing systems.

Automatic Generation of Runtime Failure Detectors from Property Templates

Fine grained error or failure detection is often indispensable for precise, effective, and efficient reactions to runtime problems. In this chapter we describe an approach that facilitates automatic generation of efficient runtime detectors for relevant classes of functional problems. The technique targets failures that commonly manifest at the boundaries between the components that form the system. It employs a model-based specification language that developers use to capture system-level properties extracted from requirements specifications. These properties are automatically translated into assertion-like checks and inserted in all relevant locations of the systems code. The main goals of our research are to define useful classes of system-level properties, identify errors and failures related to the violations of those properties, and produce assertions capable of detecting such violations. To this end we analyzed a wide range of available software specifications, bug reports for implemented systems, and other sources of information about the developers intent, such as test suites. The collected information is organized in a catalog of requirements-level property descriptions. These properties are used by developers to annotate their system design specifications, and serve as the basis for automatic assertion generation.

Model-driven generation of runtime checks for system properties

Creating runtime monitors for interesting properties is an important research problem. Existing approaches to runtime verification require specifications that not only define the property to monitor, but also contain details of the implementation, sometimes even requiring the implementation to add special variables or methods for monitoring. Often intuitive properties such as “event X should only happen when objects A and B agree” have to be translated by developers into complex specifications, for example, pre- and post-conditions on several methods that only in concert express this simple property. In most specification languages, the result of this manual translation are specifications that are so strongly tailored to the program at hand and the objects involved that, even if the property occurs again in a similar program, the whole translation process has to be repeated to create a new specification. In this paper, we introduce the concept of property templates. Property templates are pre-defined constraints that can be easily reused in specifications. They are part of a model-driven framework that translates high-level specifications into runtime monitors specialized to the problem at hand. The framework is extensible: Developers can define property templates for constraints they often need and can specialize the code generation when the default implementation is not satisfactory. We demonstrate the use of the framework in some case studies using a set of functional and structural constraints that we developed through an extensive study of existing software specifications. The key innovations of the approach we present are three. First, the properties developed with this approach are reusable and apply to a wide range of software systems, rather than being ad hoc and tailored to one particular program. Second, the properties are defined at a relatively high level of abstraction, so that no detailed knowledge of the implementation is needed to decide whether a given property applies. Third, we separate the definition of precise assertions for properties, and the use of properties. That way, experts can determine which assertions are needed to assure properties, and other developers can easily use these definitions to annotate systems.

Augmenting test suites automatically

We present an approach to augment test suites with automatically generated integration test cases. Our approach utilizes existing test cases to direct generation towards producing complex object interactions and execution sequences that have not been observed before.