Miroslav Bures

Constrained Interaction Testing: A Systematic Literature Study

Interaction testing can be used to effectively detect faults that are otherwise difficult to find by other testing techniques. However, in practice, the input configurations of software systems are subjected to constraints, especially in the case of highly configurable systems. Handling constraints effectively and efficiently in combinatorial interaction testing is a challenging problem. Nevertheless, researchers have attacked this challenge through different techniques, and much progress has been achieved in the past decade. Thus, it is useful to reflect on the current achievements and shortcomings and to identify potential areas of improvements. This paper presents the first comprehensive and systematic literature study to structure and categorize the research contributions for constrained interaction testing. Following the guidelines of conducting a literature study, the relevant data are extracted from a set of 103 research papers belonging to constrained interaction testing. The topics addressed in constrained interaction testing research are classified into four categories of constraint test generation, application, generation and application, and model validation studies. The papers within each of these categories are extensively reviewed. Apart from answering several other research questions, this paper also discusses the applications of constrained interaction testing in several domains, such as software product lines, fault detection and characterization, test selection, security, and graphical user interface testing. This paper ends with a discussion of limitations, challenges, and future work in the area.

Using AICC to create reusable adaptive hypermedia e-learning content

Adaptive hypermedia systems represent one of promising future Internet concepts, especially in the area of e-learning. One of important contemporary claims is to create reusable adaptive hypermedia content, compatible with existing systems, to ease a widespread of this promising technology. In this paper we propose method of adaptive hypermedia e-learning courses construction using AICC communication standard. This proposal is based on formal description of adaptive hypermedia course topology and its mapping to AICC communication structure

Model for Evaluation and Cost Estimations of the Automated Testing Architecture

In the automated testing, finding an optimal architecture of the scripts and level of structuring to reusable objects from an economic point of view could become a challenging task. The optimality of the architecture of automated test scripts is context dependent and many factors are playing role in the final economics of the test automation project. To support this task with an exact method, we propose a model for evaluation of the automated testing architecture. This model is based on principal structural elements, which can be identified in the automated test scripts. Elements are then composed the particular architecture and for each of them, a set of properties and metrics are defined. Using the proposed metrics, more accurate estimations of implementation and maintenance costs can be performed for the particular architecture and test automation case, especially reflecting duplicity and reuse ratio of the code.

Framework for assessment of web application automated testability

Implementation and maintenance costs of frontend automated tests depend on quality of the frontend code. By this quality we mean testability of the frontend by an automated test script and its compliance to industry standards. In the paper, we present semi-automated framework for assessment of this automated testability. The solution is based on crawling of the tested application frontend web pages and computation of set of specific metrics for frontend individual pages and the tested system. The metrics then aids automated testability assessment process to be more thorough and time efficient.

Automated testing in the Czech Republic: the current situation and issues

The paper brings an overview of the situation in automated testing in the Czech Republic by the beginning of 2014 and analyzes the trend. The results can be, under certain conditions, generalized to the Middle-European region. The analyzed data are collected from a survey of 28 Czech and Slovak professionals in the automated testing, having together 122 years of experience from 147 projects in total. In the survey we focus on the major aspects of the discipline: the economic point of view and technical state of the art.

A hybrid Q-learning sine-cosine-based strategy for addressing the combinatorial test suite minimization problem

The sine-cosine algorithm (SCA) is a new population-based meta-heuristic algorithm. In addition to exploiting sine and cosine functions to perform local and global searches (hence the name sine-cosine), the SCA introduces several random and adaptive parameters to facilitate the search process. Although it shows promising results, the search process of the SCA is vulnerable to local minima/maxima due to the adoption of a fixed switch probability and the bounded magnitude of the sine and cosine functions (from -1 to 1). In this paper, we propose a new hybrid Q-learning sine-cosine- based strategy, called the Q-learning sine-cosine algorithm (QLSCA). Within the QLSCA, we eliminate the switching probability. Instead, we rely on the Q-learning algorithm (based on the penalty and reward mechanism) to dynamically identify the best operation during runtime. Additionally, we integrate two new operations (Lévy flight motion and crossover) into the QLSCA to facilitate jumping out of local minima/maxima and enhance the solution diversity. To assess its performance, we adopt the QLSCA for the combinatorial test suite minimization problem. Experimental results reveal that the QLSCA is statistically superior with regard to test suite size reduction compared to recent state-of-the-art strategies, including the original SCA, the particle swarm test generator (PSTG), adaptive particle swarm optimization (APSO) and the cuckoo search strategy (CS) at the 95% confidence level. However, concerning the comparison with discrete particle swarm optimization (DPSO), there is no significant difference in performance at the 95% confidence level. On a positive note, the QLSCA statistically outperforms the DPSO in certain configurations at the 90% confidence level.

Exploratory testing supported by automated reengineering of model of the system under test

Exploratory Testing technique is well applicable to software development projects, where test basis is not available (or at least not complete and consistent to the extent allowing the creation of efficient test cases). The key factor for the efficiency of this technique is a structured process for the recording of explored path in the system under test. This approach also allows the creation of the test cases during exploratory testing process. These test cases can be used in the following re-testing of the system. If performed manually, the efficiency of such process strongly depends on the team organization and systematic work of the individuals in the team. This process can be aided by an automated support. In the paper, a framework, which automatically records selected tester’s actions in the system under test is presented. From these recordings, a model of the screen and action flows is reengineered and test cases are prepared. Tester is also able to define more meta-data in the test cases during this process. The recorded model and defined test cases are then available for the next rounds of testing. The performed case study shows that Exploratory Testing aided by this machine support is less resource demanding than Exploratory Testing performed manually only. Also, larger part of SUT was explored during the tests, when this systematic support was available to testers.

Transformation of IFML schemas to automated tests

Interaction Flow Modelling Language (IFML) is a recently standardized modelling language designed to capture the structure and properties of the application user interface. In the paper we discuss suitability of the IFML model for automatic generation of automated application front--end tests, as detailed model of the application front--end provides enough information to exercise particular actions in the tested application front--end to verify expected behaviour. Our approach is based on the multi--level transformation of an IFML model to automated tests. The IFML model is converted to an application front--end test model. Then abstract test cases are created from this model using set of defined rules. Finally the abstract test cases are transformed to a particular physical automated test cases using a template engine.

Internet of Things: Current Challenges in the Quality Assurance and Testing Methods

Contemporary development of the Internet of Things (IoT) technology brings a number of challenges in the Quality Assurance area. Current issues related to security, users privacy, the reliability of the service, interoperability, and integration are discussed. All these create a demand for specific Quality Assurance methodology for the IoT solutions. In the paper, we present the state of the art of this domain and we discuss particular areas of system testing discipline, which is not covered by related work sufficiently so far. This analysis is supported by results of a recent survey we performed among ten IoT solutions providers, covering various areas of IoT applications.

Model-Based Testing and Exploratory Testing: Is Synergy Possible?

Model-based testing represents an efficient method to detect software defects. This approach has a limitation, when the model of the system under test is inconsistent, incomplete, or, does not exist at all. An exploratory testing approach represents an alternative for such situations: initially it does not require a model of the tested system. The testers are using their skills and intuition to explore the system. Nevertheless, this technique has also its disadvantages. For example, it is hard to measure the effectiveness of the testing or to re-evaluate certain test when the explored path and exercised test cases are not being documented properly. In this paper, we are proposing a technique that utilizes advantages of the both model-based testing and exploratory testing and tries to eliminate their issues. The model of the application is dynamically created and updated in the exploratory stage when state data (the page user is currently on, all input, action and link elements) and every step that was taken is recorded. This structure documents what parts of the tested system were explored and what parts has to be explored later on. Recorded steps are also converted to test case scenario that can be reused later in the testing process.