Daniel Flemström

Software Components Services for Embedded Real-Time Systems

The use of software component models has become increasingly popular during the last decade, especially in the development of software for desktop applications and distributed information systems. Popular component models for these domains define basic standards for component naming, interfacing, binding, etc., in addition to standardized sets of run-time services oriented towards the application domains they target.

Vertical Test Reuse for Embedded Systems: A Systematic Mapping Study

Vertical test reuse refers to the reuse of test cases or other test artifacts over different integration levels in the software or system engineering process. Vertical test reuse has previously been proposed for reducing test effort and improving test effectiveness, particularly for embedded system development. The goal of this study is to provide an overview of the state of the art in the field of vertical test reuse for embedded system development. For this purpose, a systematic mapping study has been performed, identifying 11 papers on vertical test reuse for embedded systems. The primary result from the mapping is a classification of published work on vertical test reuse in the embedded system domain, covering motivations for reuse, reuse techniques, test levels and reusable test artifacts considered, and to what extent the effects of reuse have been evaluated.

A prototype tool for software component services in embedded real-time systems

The use of software component models has become popular during the last decade, in particular in the development of software for desktop applications and distributed information systems. However, such models have not been widely used in the domain of embedded real-time systems. There is a considerable amount of research on component models for embedded real-time systems, or even narrower application domains, which focuses on source code components and statically configured systems. This paper explores an alternative approach by laying the groundwork for a component model based on binary components and targeting the broader domain of embedded real-time systems. The work is inspired by component models for the desktop and information systems domains in the sense that a basic component model is extended with a set of services for the targeted application domain. A prototype tool for supporting these services is presented and its use illustrated by a control application.

Exploring Test Overlap in System Integration: An Industrial Case Study

Tougher safety regulations, global competition and ever increasing complexity of embedded software puts extensive pressure on the effectiveness of the software testing procedures. Previous studies have found that there exist overlaps (i.e., multiple instances of highly similar test cases) and even redundancies in the software testing process. Such overlap has been found between versions, variants and integration levels, but primarily at unit test level. Given large embedded systems involving many subsystems, does overlap exist within the system integration testing as well? In this paper, we present an industrial case study, aiming to a) evaluate if there exist test overlaps within the given context, b) if so, investigate how these overlaps are distributed, and c) find ways of reducing test effort by investigating how the knowledge of overlaps and their distribution may be used for finding candidate test cases for automation, maintenance or even removal. We have studied manual test cases, written in natural language, at a large vehicular manufacturer in Sweden. In particular, we have collected and analyzed test cases from the system integration testing levels of four different projects of a vehicle control management system. Using a similarity function, we evaluate if any overlaps between test cases exist, and where. We found that overlaps do exist within the system integration level, particularly in the form of partial test step sequences. However, very few test cases overlapped in their entirety. Some candidates for test step automation and update propagation were identified, but none for easy removal.

Similarity-based prioritization of test case automation

The importance of efficient software testing procedures is driven by an ever increasing system complexity as well as global competition. In the particular case of manual test cases at the system integration level, where thousands of test cases may be executed before release, time must be well spent in order to test the system as completely and as efficiently as possible. Automating a subset of the manual test cases, i.e, translating the manual instructions to automatically executable code, is one way of decreasing the test effort. It is further common that test cases exhibit similarities, which can be exploited through reuse when automating a test suite. In this paper, we investigate the potential for reducing test effort by ordering the test cases before such automation, given that we can reuse already automated parts of test cases. In our analysis, we investigate several approaches for prioritization in a case study at a large Swedish vehicular manufacturer. The study analyzes the effects with respect to test effort, on four projects with a total of 3919 integration test cases constituting 35,180 test steps, written in natural language. The results show that for the four projects considered, the difference in expected manual effort between the best and the worst order found is on average 12 percentage points. The results also show that our proposed prioritization method is nearly as good as more resource demanding meta-heuristic approaches at a fraction of the computational time. Based on our results, we conclude that the order of automation is important when the set of test cases contain similar steps (instructions) that cannot be removed, but are possible to reuse. More precisely, the order is important with respect to how quickly the manual test execution effort decreases for a set of test cases that are being automated.

SAGA Toolbox: Interactive Testing of Guarded Assertions

This paper presents the SAGA toolbox. It centers around development of tests, and analysis of test results, on Guarded Assertions (GA) format. Such a test defines when to test, and what to expect in such a state. The SAGA toolbox lets the user describe the test, and at the same time get immediate feedback on the test result based on a trace from the System Under Test (SUT). The feedback is visual using plots of the trace. This enables the test engineer to play around with the data and use an agile development method, since the data is already there. Moreover, the SAGA toolbox also enables the test engineer to change test stimuli plots to study the effect they have on a test. It can later generate computer programs that can feed these test stimuli to the SUT. This enables an interactive feedback loop, where immediate feedback on changes to the test, or to the test stimuli, indicate whether the test is correct and it passed or failed.

A case study of interactive development of passive tests

Testing in the active sense is the most common way to perform verification and validation of systems, but testing in the passive sense has one compelling property: independence. Independence from test stimuli and other passive tests opens up for parallel testing and off-line analysis. However, the tests can be difficult to develop since the complete testable state must be expressed using some formalism. We argue that a carefully chosen language together with an interactive work flow, providing immediate feedback, can enable testers to approach passive testing. We have conducted a case study in the automotive domain, interviewing experienced testers. The testers have been introduced to, and had hands-on practice with a tool. The tool is based on Easy Approach to Requirements Syntax (EARS) and provides an interactive work flow for developing and evaluating test results. The case study shows that i) the testers believe passive testing is useful for many of their tests, ii) they see benefits in parallelism and off-line analysis, iii) the interactive work flow is necessary for writing the testable state expression, but iv) when the testable state becomes too complex, then the proposed language is a limitation. However, the language contributes to concise tests, resembling executable requirements.