Daniel Brenner

The MORABIT Approach to Runtime Component Testing

Runtime testing is important for improving the quality of software systems. This fact holds true especially for systems which cannot be completely assembled at development time, such as mobile or ad-hoc systems. The concepts of built-in-test (BIT) can be used to cope with runtime testing, but to our knowledge there does not exist an implemented infrastructure for BIT. The MORABIT project realizes such an infrastructure and extends the BIT concepts to allow for a smooth integration of the testing process and the original business functionality execution. In this paper the requirements on the infrastructure and our solution are presented

Reducing verification effort in component-based software engineering through built-in testing

Today component- and service-based technologies play a central role in many aspects of enterprise computing. However, although the technologies used to define, implement, and assemble components have improved significantly over recent years, techniques for verifying systems created from them have changed very little. The correctness and reliability of component-based systems are still usually checked using the traditional testing techniques that were in use before components and services became widespread, and the associated costs and overheads still remain high. This paper presents an approach that addresses this problem by making the system verification process more component-oriented. Based on the notion of built-in tests (BIT)—tests that are packaged and distributed with prefabricated, off-the-shelf components—the approach partially automates the testing process, thereby reducing the level of effort needed to establish the acceptability of the system. The approach consists of a method that defines how components should be written to support and make use of run-time tests, and a resource-aware infrastructure that arranges for tests to be executed when they have a minimal impact on the delivery of system services. After providing an introduction to the principles behind component-based verification and explaining the main features of the approach and its supporting infrastructure, we show by means of a case study how it can reduce system verification effort.

Strategies for the Run-Time Testing of Third Party Web Services

Because of the dynamic way in which service-oriented architectures are configured, the correct interaction of service users and service providers can only be fully tested at run-time. However, the run-time testing of web services is complicated by the fact that they may be arbitrarily shared and may have lifetimes which are independent of the applications that use them. In this paper we investigate this situation by first identifying the different types of tests that can be applied to services at run-time and the different types of web services that can be used in service-oriented systems. We then discuss how these can be combined - identifying the combinations of tests and web services that make sense and those that do not. The resulting analysis identifies six distinct forms of run-time testing strategy of practical value in service-oriented systems.

Specifying High-Assurance Services

An enhanced approach to service specification strikes a better balance between machine processibility and human readability, and emphasizes testing- as well as reasoning-based assurance techniques. Built-in tests check the compatibility of interacting services at runtime and automatically pinpoint contract mismatches.

Ubiquitous RATs: how resource-aware run-time tests can improve ubiquitous software systems

In this paper we describe a new approach for increasing the reliability of ubiquitous software systems. This is achieved by executing tests at run-time. The individual software components are consequently accompanied by executable tests. We augment this well-known built-in test (BIT) paradigm by combining it with resource-awareness. Starting from the constraints for such resource-aware tests (RATs) we derive their design and describe a number of strategies for executing such tests under resource constraints as well as the necessary middleware. Our approach is especially beneficial to ubiquitous software systems due to their dynamic nature - which prevents a static verification of their reliability - and their inherent resource limitations.

Reducing Verification Effort in Component-Based Software Engineering through Built-In Testing

Today component- and service-based technologies play a central role in many aspects of enterprise computing. However, although the technologies used to define, implement, and assemble components have improved significantly over recent years, techniques for verifying systems created from them have changed very little. The correctness and reliability of component-based systems are still usually checked using the traditional testing techniques that were in use before components and services became widespread, and the associated costs and overheads still remain high. This paper presents an approach which addresses this problem by making the system verification process more component-oriented. Based on the notion of built-in tests - tests that are packaged and distributed with prefabricated, off-the-shelf components - the approach and supporting infrastructure help to automate some of the testing process, thereby significantly reduces system testing effort. After providing an introduction to the principles behind component-based verification, and explaining the main features of the approach, we show by means of a small example how it can reduce system verification effort

Testing Web-Services Using Test Sheets

As software services become the dominant platform for enterprise computing and B2B/B2C applications, testing their correctness and dependability assumes ever more importance. However, unlike the languages used to define and realize them, the languages used to test service-based systems have changed little over recent years. Today, tests for services and service-oriented architectures are still typically written using approaches such as xUnit or Testing and Test Control Notation (TTCN-3) developed for traditional software. While programmatic approaches allow the full power of object-oriented programming to be used to define tests, they are only intelligible to IT experts. Model-based test representation techniques such as the Unified Modeling Language (UML) testing profile and the TTCN-3 visualization features are understandable by more stakeholders but provide only partial descriptions of tests and do not scale well beyond simple algorithms. In this paper we present a new approach to software service testing which combines the expressive power of tabular tests specification techniques like Framework for Integrated Test (FIT) with programmatic techniques like xUnit and TTCN-3. The new approach also integrates test definition with test result specification and evaluation. This allows non-IT experts to define and run tests and integrates testing more tightly into the service-oriented development process.

Enabling Run-Time System Verification through Built-In Testing

Today component-based technologies play an important role in system development. However, although the technologies used to define, implement, and assemble components and systems of components have improved significantly over recent years, verification techniques have not yet caught up. Correctness and reliability of component-based systems are still often checked using traditional testing techniques. These techniques are unable to use the knowledge about the structure of the systems and its components. Therefore, an approach is needed to reduce the effort involved in system testing. Clearly, the best would be to automate large parts of the testing process. Built-in tests - tests that are packaged with components - are a good basis for such an approach