Heikki Virtanen

Automatic GUI test generation for smartphone applications - an evaluation

We present the results of an evaluation where we studied the effectiveness of automatic test generation for graphical user interface (GUI) testing of smartphone applications. To describe the context of our evaluation, the tools and the test model library we have developed for the evaluation are also presented. The library contains test models for basic S60 applications, such as camera, contacts, etc. The tools include an on-line test generator that produces sequences of so called keywords to be executed on the test targets. In our evaluation, we managed to find over 20 defects from applications that had been on the market for several months. We also describe the problems we faced during the evaluation.

Merging state-based and action-based verification

A formalism is presented that is intended to combine basic properties of both state-based and action-based verification. In state-based verification the behaviour of the system is described in terms of the properties of its states, whereas action-based methods concentrate on transitions between states. A typical state-based approach consists of representing requirements as temporal logic formulae, and model-checking the state space of the system against them. Action-based verification often consists of comparing systems according to some equivalence or preorder relation. We add state propositions to a typical process-algebraic action framework. Values of state propositions are propagated through process-algebraic compositions and reductions by augmenting actions with changes of proposition values. A modified parallel composition operator is used for synchronisation of processes and handling of state propositions. Efficient on-the-fly verification is obtained with four kinds of rejection conditions. The formalism is implemented in a new verification tool TVT.

Synthesizing Test Models from Test Cases

In this paper we describe a methodology for synthesizing test models from test cases. The context of our approach is model-based graphical user interface (GUI) testing of smartphone applications. To facilitate the deployment of model-based testing practices, existing assets in test automation should be utilized. While companies are interested in the benefits of new approaches, they may have already invested heavily in conventional test suites. The approach presented in this paper enables using such suites for creating complex test models that should have better defect detection capability. The synthesis is illustrated with examples from two small case studies conducted using real test cases from industry. Our approach is semi-automatic requiring user interaction. We also outline planned tool support to enable efficient synthesis process.

Model-Based Testing Using System vs. Test Models - What Is the Difference?

We discuss the differences between using system models and test models with respect to the model-based testing process. Although these two terms are usually used interchangeably, very little is known about the distinction between the two. System models describe internal behavior of the system under test while the test models contain the behavior from user’s or environment’s point of view. We describe how these two types of models are obtained and used throughout the model-based testing process and how they are related to each other. The discussion is based on our earlier experiences as well as on two case study examples from the telecommunication domain.

Tampere Verification Tool

Tampere Verification Tool (TVT) is a collection of programs for automated verification of concurrent and reactive systems. TVT has its roots in process algebras and explicit state space exploration, but in addition to actions, our formalism allows use of state-based information in the form of truth-valued state propositions. Furthermore, it contains three types of state proposition-like notions to support on-the-fly verification, and one state proposition to exploit partially defined processes. TVT supports compositional state space construction, stubborn sets and visual verification.

A Comparison of Test Generation Algorithms for Testing Application Interactions

Testing the interactions of different applications running in the same operating system or platform poses challenges for manual testing and conventional script-based automation. Towards this end, we have developed an online model-based testing solution allowing efficient testing of such interactions. This paper presents the results of the comparison of algorithms used for generating tests for interaction testing. The comparison is based on our experiments with a number of different algorithms as well as results from the earlier studies by others. Given the simplicity of implementation, Random Walk seems very useful and practical solution for online test generation. However, each of the compared algorithms has its strong points, making the selection dependent of the metric one wants to emphasize and the available a priori information. Especially when the execution of test events is slow, smarter algorithms have advantage over simple random walk.

Context-Sensitive Visibility

An improvement to the so-called visual verification approach is presented. Visual verification is a method for checking the correctness of the behaviour of a reactive or concurrent system. It shares a great deal of common ground with ordinary formal state space verification, but is more user-friendly. This is because the user does not need to specify in detail the properties that the system must satisfy to be correct. Instead, the user only lists the atomic actions that are relevant for the property. Computer tools are used to obtain a graphical representation which is a summary of all possible alternative behaviours of the system, and the user then analyses the result. The improvement presented in this article allows the user to pick a region of the graphical representation and investigate it in more detail, without being overwhelmed by the details outside the region. The improvement is illustrated by analysing the livelocks in a model of the alternating bit protocol.

Verification of Safety-Critical Systems: A Case Study Report on Using Modern Model Checking Tools

paper, we describe a case study where a simple 2oo3 voting scheme for a shutdown system was verified using two bounded model checking tools, CBMC and EBMC. The system represents Systematic Capability level 3 according to IEC 61508 ed2.0. The verification process was based on requirements and pseudo code, and involved verifying C and Verilog code implementing the pseudo code. The results suggest that the tools were suitable for the task, but require considerable ntraining to reach productive use for code embedded in industrial equipment. We also identified some issues in the development process that could be streamlined with the use of more formal verification methods. Towards the end of the paper, we discuss the issues we found and how to address them in a practical setting.