Antti Kervinen

Making model-based testing more agile: a use case driven approach

We address the problem of misalignment of artifacts developed in agile software development projects and those required by model-based test generation tools. Our solution is domain specific and relies on the existence of domain experts to design the test models. The testers interface the test generation systems with use cases that are converted into sequences of so called action words corresponding to user events at a high level of abstraction. To support this scheme, we introduce a coverage language and an algorithm for automatic test generation.

Towards Deploying Model-Based Testing with a Domain-Specific Modeling Approach

Model-based testing, automating the generation of test cases is technically superior to conventional scripted testing. However, there have been difficulties in deploying the methodology in large scale industrial context. In this paper we present a domain-specific approach to model-based GUI testing that should be easier to adopt than more generic solutions. The method is based on keywords and action words that are considered as best practices in conventional GUI test automation. The basic idea is to record GUI events just like in capture/replay tools, but instead of producing scripts that can be hard to maintain, we produce sequences of keywords. These sequences are further transformed semi-automatically into labeled transition systems where action words are used as transition labels. The action words model user behavior at a high level of abstraction while the keywords correspond to the GUI navigation. We also describe the associated tool set that we are developing and an example of using the approach

Model-Based Testing Service on the Web

Model-based testing (MBT) seems to be technically superior to conventional test automation. However, MBT features some difficulties that can hamper its deployment in industrial contexts. We are developing a domain-specific MBT solution for graphical user interface (GUI) testing of Symbian S60 smartphone applications. We believe that such a tailor-made solution can be easier to deploy than ones that are more generic. In this paper, we present a service concept and an associated web interface that hide the inherent complexity of the test generation algorithms and large test models. The interface enables an easy-to-use MBT service based on the well-known keyword concept. With this solution, a better separation of concerns can be obtained between the test modeling tasks that often require special expertise, and test execution that can be performed by testers. We believe that this can significantly speed up the industrial transfer of model-based testing technologies, at least in this context.

Automatic GUI test generation for smartphone applications - an evaluation

We present the results of an evaluation where we studied the effectiveness of automatic test generation for graphical user interface (GUI) testing of smartphone applications. To describe the context of our evaluation, the tools and the test model library we have developed for the evaluation are also presented. The library contains test models for basic S60 applications, such as camera, contacts, etc. The tools include an on-line test generator that produces sequences of so called keywords to be executed on the test targets. In our evaluation, we managed to find over 20 defects from applications that had been on the market for several months. We also describe the problems we faced during the evaluation.

Model-Based testing through a GUI

So far, model-based testing approaches have mostly been used in testing through various kinds of APIs. In practice, however, testing through a GUI is another equally important application area, which introduces new challenges. In this paper, we introduce a new methodology for model-based GUI testing. This includes using Labeled Transition Systems (LTSs) in conjunction with action word and keyword techniques for test modeling. We have also conducted an industrial case study where we tested a mobile device and were able to find previously unreported defects. The test environment included a standard MS Windows GUI testing tool as well as components implementing our approach. Assessment of the results from an industrial point of view suggests directions for future development.

Alphabet-Based Synchronisation is Exponentially Cheaper

We study the complexity of verification problems in which a preorder relation between an implementation and a specification is checked, when the specification is given as a parallel composition of processes. This problem turns out to be PSPACE- or EXPSPACE complete, depending on the type of the parallel composition operator that is used in the construction of the specification. This implies that confusion with different parallel composition operators may lead to erroneous complexity claims. We fix one such erroneous result presented in an earlier publication. We also show that the application of hiding, renaming or just one interleaving parallel composition operation to a specification for which the problem is in PSPACE, may raise the complexity of the problem to EXPSPACE-hard.

Synthesizing Test Models from Test Cases

In this paper we describe a methodology for synthesizing test models from test cases. The context of our approach is model-based graphical user interface (GUI) testing of smartphone applications. To facilitate the deployment of model-based testing practices, existing assets in test automation should be utilized. While companies are interested in the benefits of new approaches, they may have already invested heavily in conventional test suites. The approach presented in this paper enables using such suites for creating complex test models that should have better defect detection capability. The synthesis is illustrated with examples from two small case studies conducted using real test cases from industry. Our approach is semi-automatic requiring user interaction. We also outline planned tool support to enable efficient synthesis process.

Heuristics for Faster Error Detection With Automated Black Box Testing

Three building blocks for test guidance algorithms, the step evaluation, the state evaluation and the evaluation order, are proposed in this paper. We show how a simple family of coverage criteria can be used to evaluate individual testing steps, and how the nondeterministic behaviour of the tested system can be handled and longer term test step plans created with the state evaluation. We use the evaluation order to de ne which and when states are evaluated. Six heuristic algorithms based on these ideas are implemented. Four of them use a game-like approach to black box testing. In addition, three other test guidance algorithms are implemented for comparison. The algorithms are compared by measuring the number of testing steps required for detecting errors that are in ltrated to the conference protocol systems of two different sizes.

Creating a Test Model Library for GUI Testing of Smartphone Applications (Short Paper)

Smartphones are becoming increasingly complex, and the interactions between the different applications make testing even more difficult given the time-to-market pressures and the limits of current test automation systems. Towards these ends, we have built an open source test model library for Symbian S60 GUI testing. This paper describes and analyzes our experiences in building the library.

Controlling Testing Using Three-Tier Model Architecture

Abstract In this paper, based on our earlier work, we introduce test model architecture for model-based testing through a GUI. The model architecture consists of three tiers. The tiers separate important concerns in GUI testing: navigation in the GUI using keywords, high-level concepts expressed as action words, and test-control related issues defined as control words. For test control, we define a novel coverage language to express coverage objectives. Furthermore, we introduce our refined vision for the associated tool platform enabling system level testing in the Symbian environment. The architecture includes a commercial GUI testing tool that we have extended with components enabling the use of test models.