Alberto González

A Model for the Measurement of the Runtime Testability of Component-Based Systems

Runtime testing is emerging as the solution for the integration and validation of software systems where traditional development-time integration testing cannot be performed, such as Systems of Systems or Service Oriented Architectures.However, performing tests during deployment or in-service time introduces interference problems, such as undesired side-effects in the state of the system or the outside world.This paper presents a qualitative model of runtime testability that complements Binders classical testability model, and a generic measurement framework for quantitatively assessing the degree of runtime testabilityof a system based on the ratio of what can be tested at runtime vs. what would have been tested during development time.A measurement is devised for the concrete case of architecture-based test coverage, by using a graph model of the systems architecture. Concretely, two testabilitystudies are performed for two component based systems, showing how to measure the runtime testability of a system.

Architecture support for runtime integration and verification of component-based Systems of Systems

Systems-of-systems (SoS) represent a novel kind of system, for which runtime evolution is a key requirement, as components join and leave during runtime. Current component integration and verification techniques are not enough in such a dynamic environment. In this paper we present ATLAS, an architectural framework that enables the runtime integration and verification of a system, based on the built-in test paradigm. ATLAS augments components with two specific interfaces to add and remove tests, and to provide adequate testability features to run these tests. To illustrate our approach, we present a case study of a dynamic reconfiguration scenario of components, in the maritime safety and security domain, using our implementation of ATLAS for the fractal component model. We demonstrate that built-in testing can be extended beyond development-time component integration testing, to support runtime reconfiguration and verification of component-based systems.

Using Fault Screeners for Software Error Detection

Fault screeners are simple software (or hardware) constructs that detect variable value errors based on unary invariant checking. In this paper we evaluate and compare the performance of three low-cost screeners (Bloom filter, bitmask, and range screener) that can be automatically integrated within a program, and trained during the testing phase. While the Bloom filter has the capacity of retaining virtually all variable values associated with proper program execution, this property comes with a much higher false positive rate per unit of training effort, compared to the more simple range and bitmask screeners, that compresses all value information in terms of a single lower and upper bound or a bitmask, respectively. We present a novel analytic model that predicts the false positive and false negative rate for ideal (i.e., screeners that store each individual value during training) and simple (e.g., range and bitmask) screeners. We show that the model agrees with our empirical findings. Furthermore, we describe an application of all screeners, where the screener’s error detection output is used as input to a fault localization process that provides automatic feedback on the location of residual program defects during deployment in the field.