Erik Kamsties

Model-based system testing of software product families

In software product family engineering reusable artifacts are produced during domain engineering and applications are built from these artifacts during application engineering. Modeling variability of current and future applications is the key for enabling reuse. The proactive reuse leads to a reduction in development costs and a shorter time to market. Up to now, these benefits have been realized for the constructive development phases, but not for testing. This paper presents the ScenTED technique (Scenario based TEst case Derivation), which aims at reducing effort in product family testing. ScenTED is a model-based, reuse-oriented technique for test case derivation in the system test of software product families. Reuse of test cases is ensured by preserving variability during test case derivation. Thus, concepts known from model-based testing in single system engineering, e.g., coverage metrics, must be adapted. Experiences with our technique gained from an industrial case study are discussed and prototypical tool support is illustrated.

Testing Variabilities in Use Case Models

The derivation of system test cases for product families is difficult due to variability in the requirements, since each variation point multiplies the number of possible behaviors to be tested. This paper proposes an approach to develop domain test cases from use cases that contain variabilities and to derive application test cases from them. The basic idea to avoid combinatorial explosion is to preserve the variability in domain test cases. New strategies to capture variability in test cases are suggested, which in combination help dealing with all basic types of variability in a use case and in its relationships (e.g., include >>).

The syntactically dangerous all and plural in specifications

The items on the private check list are specific problems involving the correct use of the natural language in which the RS is written. It includes incorrect grammar, incorrect word placement, and all kinds of ambiguities. The lists of grammatical and word-placement problems are similar despite the difference in the natural languages involved. The syntactic problems are symptoms of ambiguities in meaning - a grammatical problem occurs when part of a sentence disagrees with another, and each choice in the disagreement corresponds to a different meaning. The use of plural to describe a property of elements of a set or of sets makes it difficult to determine whether the property is that of each element or of the whole set. A specification inspector can certainly search for plural constructions in a specification to examine each for its danger. Best of all is for a specification writer not to write plural statements when describing properties of each element of a set.

A controlled experiment to evaluate how styles affect the understandability of requirements specifications

Abstract This paper presents a controlled experiment in which two different requirements specification styles (white-box and black-box) were compared concerning the understandability of two requirements specifications from the viewpoint of a customer. The results of the experiment confirm the common belief that black-box requirements specifications (e.g., documented with SCR) are easier to understand from a customer point of view than white-box specifications (e.g., documented with UML). Questions about particular functions and behavior of the specified system were answered faster and more correctly by the participants. This result suggests that using a black-box specification style when communicating with customers is beneficial.

Considering variability in a system family's architecture during COTS evaluation

COTS (commercial off-the-shelf) component designers and developers often envision different usage contexts for their component and, therefore, provide it with adaptation possibilities. These adaptation possibilities are especially important when considering system families. System family engineering is currently an emerging discipline. Variability is a core property of system families which allows deriving different customer-specific applications from a core artifact base. A system familys core artifact base may also be populated with COTS components. These COTS components then need to support the system familys variability, i.e. they have to offer the possibility to adapt them to different customer-specific applications. Through their adaptation possibilities COTS components are able to meet this requirement. During COTS evaluation, a system familys requirements and architecture need to be taken into account. Variability is inherent in both. That is, the question is how to evaluate COTS with regard to variable features. In this paper, we describe variability in architecture in more detail and point out how this variability needs to be reflected in COTS evaluation criteria. The contribution is an extension of ‘traditional COTS evaluation criteria in order to consider a system familys variability.

Software Product Family Evaluation

This paper proposes a 4-dimensional software product family engineering evaluation model. The 4 dimensions relate to the software engineering concerns of business, architecture, organisation and process. The evaluation model is meant to be used within organisations to determine the status of their own software family engineering. Evaluation results may be used for benchmarking or improvement plans.

Szenario-basierter Systemtest von Software-Produktfamilien

Zusammenfassung In der Produktfamilienentwicklung werden durch zwei Entwicklungsprozesse, Domain und Application Engineering, zunachst wiederverwendbare Entwicklungsartefakte produziert, um diese anschließend zur Konstruktion von kundenspezifischen Applikationen einzusetzen. Die Wiederverwendbarkeit wird durch die explizite Definition der Variabilitat der geplanten Applikationen einer Produktfamilie erzielt. Diese proaktive Wiederverwendung ist bisher in den konstruktiven Entwicklungsphasen realisiert, jedoch noch nicht im Test. Mit ScenTED (Scen_ario based TE_st Case D_erivation) wird in diesem Beitrag eine wiederverwendungsorientierte Technik zur Testfallerstellung fur den Systemtest, dem Test eines ausfuhrbaren Systems gegen spezifizierte Use-Cases, von Produktfamilien vorgestellt. ScenTED basiert auf zwei Kernideen: der Erhaltung der Variabilitat in Testfallen und der Szenario-basierten Verfeinerung der Testfalle. Durch die Erhaltung der Variabilitat wird die Wiederverwendbarkeit von Testfallen gesichert; die Szenario-basierte Verfeinerung ermoglicht Nachvollziehbarkeit durch die durchgangige Nutzung von Szenarien. In diesem Beitrag wird die ScenTED-Technik an einem Beispiel erlautert und Erfahrungen aus dem industriellen Einsatz diskutiert. AbstractProduct family engineering consists of two development processes: Domain engineering and application engineering. Reuseable artefacts are created in the domain engineering process. These artefacts are used for the creation of customer specific applications during the application engineering process. Reusability is achieved with the explicit definition of variability that specifies the potential applications of the product family. The proactive reuse has been realized in the construnction phases of the development process, but not within the test phases. We propose ScenTED (Scen_ario based TE_st Case D_erivation) for the creation of reusable test cases for a use case based system testing in product family engineering. ScenTED has two key ideas: The first is the preservation of variability in test cases and the second idea is the scenario-based refinement of test cases. The preservation of variability ensures the reusability of test cases. The scenario-based refinement enables the traceability from requirements to test cases. We present the ScenTED technique in this article with an example and discuss an industrial case study.

Guest editorial: 10th anniversary workshop on Requirements Engineering: Foundation for Software Quality (REFSQ’2004)

For the tenth time, the international workshop Requirements Engineering: Foundation for Software Quality (REFSQ) gathered researchers from around the world. The tenth anniversary was celebrated in the beautiful city of Riga, Latvia on the 6th and 7th of June, 2004, in connection with the International Conference on Advanced Information Systems Engineering (CAiSE’2004). The anniversary workshop had 27 participants from both academia and industry representing 15 countries from four continents. The REFSQ workshop series provides an annual, highly interactive forum for ground-breaking research in RE. The original aim of REFSQ was to foster research in requirements engineering with special focus on its links to the overall quality of the resulting systems. Over the years, the scope of the workshop has grown, both encompassing different and more diverse type of ‘‘systems’’ (from the initial software systems to business processes and socio-technical systems), and extending to other issues not necessarily focused on quality. The format of the workshop, with its strong emphasis on multifaceted, in depth discussions among participants over papers presented, has been highly successful in promoting active involvement and encouraging the emergence of innovative ideas. REFSQ invites publication and dialog regarding new solutions to known RE issues, ideas that are likely to initiate new threads inRE research, industrial problem statements, and generalisations from industrial experiences. The workshop also aims at intensive interaction between its participants and provokes discussion by guiding presentations, individual paper discussions, as well as topic discussions at the end of each session. This year, in addition to standard presentations, two special sessions devoted to a set of invited ‘‘anniversary papers’’ were included in the program: the first one on quality issues in RE and their relevance in industry; the second one on retrospective studies on the research output of REFSQ since its inception—as well as an outlook on future research. The REFSQ’2004 Call for Papers invited submissions on a wide range of RE issues, such as: understanding and improving RE-processes; new methods and method engineering approaches for RE; evaluations of RE methods, techniques and tools; empirical studies of industrial RE practice; as well as transdisciplinary theories of and paradigms for RE. This special issue includes four extended, revised and rigorously re-reviewed papers selected among those presented at REFSQ’04 (original versions are available in the workshop proceedings [1]). The selected best papers cover topics including the role of requirements in negotiation processes and the generation of requirements through novel elicitation techniques. A wide range of interesting results and important directions of further research are provided. Finally a summary of the anniversary events at REFSQ’04 (the anniversary papers can be downloaded from http://www.refsq.org) with focus on retrospectives and outlooks on RE research, and a summary of the best papers included in this special issue are provided.