Oliviero Riganelli

AutoBlackTest: Automatic Black-Box Testing of Interactive Applications

Automatic test case generation is a key ingredient of an efficient and cost-effective software verification process. In this paper we focus on testing applications that interact with the users through a GUI, and present AutoBlackTest, a technique to automatically generate test cases at the system level. AutoBlackTest uses reinforcement learning, in particular Q-Learning, to learn how to interact with the application under test and stimulate its functionalities. The empirical results show that AutoBlackTest can execute a relevant portion of the code of the application under test, and can reveal previously unknown problems by working at the system level and interacting only through the GUI.

AutoBlackTest: a tool for automatic black-box testing

In this paper we present AutoBlackTest, a tool for the automatic generation of test cases for interactive applications. AutoBlackTest interacts with the application though its GUI, and uses reinforcement learning techniques to understand the interaction modalities and to generate relevant testing scenarios. Early results show that the tool has the potential of automatically discovering bugs and generating useful system and regression test suites.

Link: exploiting the web of data to generate test inputs

Applications that process complex data, such as maps, personal data, book information, travel data, etc., are becoming extremely common. Testing such applications is hard, because they require realistic and coherent test inputs that are expensive to generate manually and difficult to synthesize automatically. So far the research on test case generation techniques has focused mostly on generating test sequences and synthetic test inputs, and has payed little attention to the generation of complex test inputs. This paper presents Link, a technique to automatically generate test cases for applications that process complex data. The novel idea of Link is to exploit the Web of Data to generate test data that match the semantics of the related fields, and satisfy the semantic constraints that arise among interrelated fields. Link automatically analyzes the GUI of the application under test, generates a model of the required inputs, queries DBPedia to extract the data that can be used in the tests, and uses the extracted data to generate complex system test inputs. The experimental results show that Link can generate realistic and coherent test inputs that can exercise behaviors difficult to exercise with currently available techniques.

Automatic testing of GUI-based applications

Testing GUI‐based applications is hard and time consuming because it requires exploring a potentially huge execution space by interacting with the graphical interface of the applications. Manual testing can cover only a small subset of the functionality provided by applications with complex interfaces, and thus, automatic techniques are necessary to extensively validate GUI‐based systems. This paper presents AutoBlackTest, a technique to automatically generate test cases at the system level. AutoBlackTest uses reinforcement learning, in particular Q‐learning, to learn how to interact with the application under test and stimulate its functionalities. When used to complement the activity of test designers, AutoBlackTest reuses the information in the available test suites to increase its effectiveness. The empirical results show that AutoBlackTest can sample better than state of the art techniques the behaviour of the application under test and can reveal previously unknown problems by working at the system level and interacting only through the graphical user interface. Copyright

SEIM: static extraction of interaction models

The quality of systems that integrate Web services provided by independent organizations depends on the ways the systems interact with the services, i.e., on their interaction protocols, which are not always easy to deduce by inspecting the code. Accurate models of interaction protocols provide a comprehensive view of the interactions, and support manual and automatic analysis of corner cases that are often difficult to discover, and are responsible from many subtle failures. Models of program behavior can be extracted either statically from the source code or dynamically from execution traces. Dynamic techniques cannot reveal behaviors that have not been executed, and thus dynamic models are partially useful to identify critical corner cases. Static techniques produce models of the whole execution space, and thus static models are more useful to find critical cases. Most static techniques to extract models of the code behavior focus on the usage protocol, i.e., how the services can be used, rather than the interaction protocol, i.e., how applications use services. While usage protocols can help software engineers understanding the service behavior, interaction protocols describe the subset of actual interactions between the system and the integrated services, and provide the information required to understand critical interactions. In this paper, we present SEIM, a technique to extract models of the service interaction protocol. SEIM extracts models statically, and includes a novel refinement strategy to eliminate infeasible behaviors that reduce the usability of statically derived models. This paper describes the technique and provides some empirical data from the early experience of SEIM with an application that interacts with the eBay Web Services.

Extracting widget descriptions from GUIs

Graphical User Interfaces (GUIs) are typically designed to simplify data entering, data processing and visualization of results. However, GUIs can also be exploited for other purposes. For instance, automatic tools can analyze GUIs to retrieve information about the data that can be processed by an application. This information can serve many purposes such as ease application integration, augment test case generation, and support reverse engineering techniques. In the last years, the scientific community provided an increasing attention to the automatic extraction of information from interfaces. For instance, in the domain of Web applications, learning techniques have been used to extract information from Web forms. The knowledge about the data that can be processed by an application is not only relevant for the Web, but it is also extremely useful to support the same techniques when applied to desktop applications. In this paper we present a technique for the automatic extraction of descriptive information about the data that can be handled by widgets in GUI-based desktop applications. The technique is grounded on mature standards and best practices about the design of GUIs, and exploits the presence of textual descriptions in the GUIs to automatically obtain descriptive data for data widgets. The early empirical results with three desktop applications show that the presented algorithm can extract data with high precision and recall, and can be used to improve generation of GUI test cases.

Policy enforcement with proactive libraries

Software libraries implement APIs that deliver reusable functionalities. To correctly use these functionalities, software applications must satisfy certain correctness policies, for instance policies about the order some API methods can be invoked and about the values that can be used for the parameters. If these policies are violated, applications may produce misbehaviors and failures at runtime. Although this problem is general, applications that incorrectly use API methods are more frequent in certain contexts. For instance, Android provides a rich and rapidly evolving set of APIs that might be used incorrectly by app developers who often implement and publish faulty apps in the marketplaces. To mitigate this problem, we introduce the novel notion of proactive library, which augments classic libraries with the capability of proactively detecting and healing misuses at runtime. Proactive libraries blend libraries with multiple proactive modules that collect data, check the correctness policies of the libraries, and heal executions as soon as the violation of a correctness policy is detected. The proactive modules can be activated or deactivated at runtime by the users and can be implemented without requiring any change to the original library and any knowledge about the applications that may use the library. We evaluated proactive libraries in the context of the Android ecosystem. Results show that proactive libraries can automatically overcome several problems related to bad resource usage at the cost of a small overhead.

Verifying Policy Enforcers

Policy enforcers are sophisticated runtime components that can prevent failures by enforcing the correct behavior of the software. While a single enforcer can be easily designed focusing only on the behavior of the application that must be monitored, the effect of multiple enforcers that enforce different policies might be hard to predict. So far, mechanisms to resolve interferences between enforcers have been based on priority mechanisms and heuristics. Although these methods provide a mechanism to take decisions when multiple enforcers try to affect the execution at a same time, they do not guarantee the lack of interference on the global behavior of the system. In this paper we present a verification strategy that can be exploited to discover interferences between sets of enforcers and thus safely identify a-priori the enforcers that can co-exist at run-time. In our evaluation, we experimented our verification method with several policy enforcers for Android and discovered some incompatibilities.

Healing Data Loss Problems in Android Apps

Android apps should be designed to cope with stop-start events, which are the events that require stopping and restoring the execution of an app while leaving its state unaltered. These events can be caused by run-time configuration changes, such as a screen rotation, and by context-switches, such as a switch from one app to another. When a stop-start event occurs, Android saves the state of the app, handles the event, and finally restores the saved state. To let Android save and restore the state correctly, apps must provide the appropriate support. Unfortunately, Android developers often implement this support incorrectly, or do not implement it at all. This bad practice makes apps to incorrectly react to stop-start events, thus generating what we defined data loss problems, that is Android apps that lose user data, behave unexpectedly, and crash due to program variables that lost their values. Data loss problems are difficult to detect because they might be observed only when apps are in specific states and with specific inputs. Covering all the possible cases with testing may require a large number of test cases whose execution must be checked manually to discover whether the app under test has been correctly restored after each stop-start event. It is thus important to complement traditional in-house testing activities with mechanisms that can protect apps as soon as a data loss problem occurs in the field. In this paper we present DataLossHealer, a technique for automatically identifying and healing data loss problems in the field as soon as they occur. DataLossHealer is a technique that checks at run-time whether states are recovered correctly, and heals the app when needed. DataLossHealer can learn from experience, incrementally reducing the overhead that is introduced avoiding to monitor interactions that have been managed correctly by the app in the past.

Holistic development of online public services

eGovernment initiatives have to take optimum advantage of the opportunities offered by the use modern technologies increasing productivity and improving operational efficiencies while leading to reduced costs. However, many eGovernment initiatives are unsuccessful leading to huge loss of money within the Public Sector and also to loss of trust on new technologies from the side of stakeholders. The online public service delivery brings with it a whole nexus of problems that cannot be solved by technological means alone. The technical perspective can fails to bear fruit. In developing and maintaining online public services and in order to encourage a climate of cooperation it is vital to redefine existing working practices considering the socio-organizational relationships in which they take place. In this paper, we propose a model of abstract and functional dimensions that affect the success of eGovernment initiatives. On the basis of our model, necessary short life cycle solutions must evolve towards a final solution that allows a reliable and flexible accomplishment of modernization projects. In this context, our contribution could be used as a metaphor to represent the necessary management of eGovernment initiatives.