Patrick Harms

AutoQUEST -- Automated Quality Engineering of Event-Driven Software

In this paper, we present AutoQUEST, a testing platform for Event-Driven Software (EDS) that decouples the implementation of testing techniques from the concrete platform they should be applied to. AutoQUEST provides the means to define testing techniques against an abstract Application Programming Interface (API) and provides plugins to port the testing techniques to distinct platforms. The requirements on plug-in implementations for AutoQUEST are kept low to keep the porting effort low. We implemented several testing techniques on top of AutoQUEST and provide five plugins for concrete software platforms, which demonstrates the capabililities of our approach.

Usage-Based Automatic Detection of Usability Smells

With an increasing number of supported devices, usability evaluation of websites becomes a laborious task. Therefore, usability evaluation should be automated as far as possible. In this paper, we present a summative method for automated usability evaluation of websites. The approach records user actions and transforms them into task trees. The task trees are then checked for usability smells to identify potential usability issues. The approach was applied in two case studies and shows promising results in the identification of four types of usability smells.

The MIDAS Cloud Platform for Testing SOA Applications

While Service Oriented Architectures (SOAs) are for many parts deployed online, and today often in a cloud, the testing of the systems still happens mostly locally. In this paper, we want to present the MIDAS Testing as a Service (TaaS), a cloud platform for the testing of SOAs. We focus on the testing of whole SOA orchestrations, a complex task due to the number of potential service interactions and the increasing complexity with each service that joins an orchestration. Since traditional testing does not scale well with such a complex setup, we employ a Model-based Testing (MBT) approach based on the Unified Modeling Language (UML) and the UML Testing Profile (UTP) within MIDAS. Through this, we provide methods for functional testing, security testing, and usage-based testing of service orchestrations. Through harnessing the computational power of the cloud, MIDAS is able to generate and execute complex test scenarios which would be infeasible to run in a local environment.

Combining usage-based and model-based testing for service-oriented architectures in the industrial practice

Usage-based testing focuses quality assurance on highly used parts of the software. The basis for this are usage profiles based on which test cases are generated. There are two fundamental approaches in usage-based testing for deriving usage profiles: either the system under test (SUT) is observed during its operation and from the obtained usage data a usage profile is automatically inferred, or a usage profile is modeled by hand within a model-based testing (MBT) approach. In this article, we propose a third and combined approach, where we automatically infer a usage profile and create a test data repository from usage data. Then, we create representations of the generated tests and test data in the test model from an MBT approach. The test model enables us to generate executable Testing and Test Control Notation version 3 (TTCN-3) and thereby allows us to automate the test execution. Together with industrial partners, we adopted this approach in two pilot studies. Our findings show that usage-based testing can be applied in practice and greatly helps with the automation of tests. Moreover, we found that even if usage-based testing is not of interest, the incorporation of usage data can ease the application of MBT.

Consistency of Task Trees Generated from Website Usage Traces

Task trees are an established method for modeling the usage of a website as required to accomplish user tasks. They define the necessary actions and the order in which users need to perform them to reach a certain goal. Modeling task trees manually can be a laborious task, especially if a website is rather complex. In previous work, we presented a methodology for automatically generating task trees based on recorded user actions on a website. We did not verify, if the approach generates similar results for different recordings of the same website. Only if this is given, the task trees can be the basis for a subsequent analysis of the usage of a website, e.g., a usability analysis. In this paper, we evaluate our approach in this respect. For this, we generated task trees for different sets of recorded user actions of the same website and compared the resulting task trees. Our results show, that the generated task trees are consistent but that the level of consistency depends on the type of website or the ratio of possible to recorded actions on a website.

The Quality in Quantity-Enhancing Text-based Research

Computers are becoming more and more a tool for researchers in the humanities. There are already several projects which aim to implement environments and infrastructures to support research. However, they either address qualitative or quantitative research methods, and there has been less work considering support for both methodologies in one environment. This paper analyzes the difference between qualitative and quantitative research in the humanities, outlines some examples and respective projects, and states why the support for both methodologies needs to be combined and how it might be used to form an integrated research infrastructure for the humanities.

Representativeness and Descriptiveness of Task Trees Generated from Website Usage Traces

Task trees are often used to define the actions on a software as well as their order which is required to accomplish a certain task. With an increasing task complexity, their creation can be laborious and error-prone. Hence, there was work done to generate them automatically from recordings of user actions. In this paper, we assess for one of these approaches if the generated task trees are representative and descriptive for recorded and also unrecorded user actions. This characteristic is important as it allows for subsequent valid analyses of the software usage based on these task trees. For our evaluations, we transform the task trees generated from one set of recorded actions into grammars for the language spoken between the user and the software. From these grammars, we generate parsers with which we try to parse action combinations in other usage recordings. Our results show, that the approach under analysis produces partially representative task trees, which are also descriptive for unrecorded user behavior.