Alessandra Bagnato

Evaluating the TESTAR tool in an industrial case study

[Context] Automated test case design and execution at the GUI level of applications is not a fact in industrial practice. Tests are still mainly designed and executed manually. In previous work we have described TESTAR, a tool which allows to set-up fully automatic testing at the GUI level of applications to find severe faults such as crashes or non-responsiveness. [Method] This paper aims at the evaluation of TESTAR with an industrial case study. The case study was conducted at SOFTEAM, a French software company, while testing their Modelio SaaS system, a cloud-based system to manage virtual machines that run their popular graphical UML editor Modelio. [Goal] The goal of the study was to evaluate how the tool would perform within the context of SOFTEAM and on their software application. On the other hand, we were interested to see how easy or difficult it is to learn and implant our academic prototype within an industrial setting. [Results] The effectiveness and efficiency of the automated tests generated with TESTAR can definitely compete with that of the manual test suite. [Conclusions] The training materials as well as the user and installation manual of TESTAR need to be improved using the feedback received during the study. Finally, the need to program Java-code to create sophisticated oracles for testing created some initial problems and some resistance. However, it became clear that this could be solved by explaining the need for these oracles and compare them to the alternative of more expensive and complex human oracles. The need to raise consciousness that automated testing means programming solved most of the initial problems.

The OMG UML Testing Profile in Use--An Industrial Case Study for the Future Internet Testing

The EU funded FITTEST FP7 project aims to address the Future Internet (FI) testing challenges. FITTEST will be integrated in three pilot applications provided by three industrial partners, IBM, Sulake and Soft am. This paper presents the Modelio SaaS product and case study context selected by Soft am as FITTEST Project industrial application and the usage of the Object Management Group (OMG) UML Testing Profile module. In the paper, researchers present the advanced software engineering methods proposed by FITTEST and the usage of the OMG UML Testing Profile (UTP) in a real industrial environment within Soft am Modelio SaaS.

Combinatorial Testing Tool Learnability in an Industrial Environment

[Context] Numerous combinatorial testing techniques are available for generating test cases. However, many of them are never used in practice. [Objective] Considering that learn ability plays a vital role in initial adoption or rejection of a technology, in this paper we aim to investigate the learnability of a combinatorial testing tool in an industrial environment. [Method] A case study research method was designed and conducted, by including i) the definition of learnability measures for test cases models built using a combinatorial testing tool. ii) A training program was also implemented. iii) Qualitative and quantitative evaluation based on a three-level strategy was carried out (Reaction, Learning, and Performance). [Results] At the first level, the tool was perceived as easy to learn by the trainees (from a five-point ordinal scale). However, at the second level, during hands-on learning, it changed slightly: According to the working diaries, there were major difficulties. At third level, analyzing the learning curve of each trainee, we observe that semantic errors made per each subject were reduced slightly over the time.

FITTEST: A new continuous and automated testing process for future Internet applications

Since our society is becoming increasingly dependent on applications emerging on the Future Internet, quality of these applications becomes a matter that cannot be neglected. However, the complexity of the technologies involved in Future Internet applications makes testing extremely challenging. The EU FP7 FITTEST project has addressed some of these challenges by developing and evaluating a Continuous and Integrated Testing Environment that monitors a Future Internet application when it runs such that it can automatically adapt the testware to the dynamically changing behaviour of the application.

Integration of a graph-based model indexer in commercial modelling tools

Softeam has over 20 years of experience providing UML-based modelling solutions, such as its Modelio modelling tool, and its Constellation enterprise model management and collaboration environment. Due to the increasing number and size of the models used by Softeams clients, Softeam joined the MONDO FP7 EU research project, which worked on solutions for these scalability challenges and produced the Hawk model indexer among other results. This paper presents the technical details and several case studies on the integration of Hawk into Softeams toolset. The first case study measured the performance of Hawks Modelio support using varying amounts of memory for the Neo4j backend. In another case study, Hawk was integrated into Constellation to provide scalable global querying of model repositories. Finally, the combination of Hawk and the Epsilon Generation Language was compared against Modelio for document generation: for the largest model, Hawk was two orders of magnitude faster.

How Does the UML Testing Profile Support Risk-Based Testing

The increasing complexity of software-intensive systems raises a lot of challenges demanding new techniques for ensuring their overall quality. The risk of not meeting the expected level of quality has negative impact on business, customers, environment and people, especially in the context of safety/security-critical systems. The importance of risk assessment, analysis and management has been well understood both in the literature and practice, which has led to the definition of a number of well-known standards. In the recent years, Risk-Based Testing (RBT) is gaining more attention, especially focusing on test prioritization and selection based on risks. On the other hand, model-based testing (MBT) provides a systematic and automated way to facilitate rigorous testing of software-intensive systems. MBT has been an intense area of research and a large number of MBT techniques have been developed in literature and practice in the last decade. In this paper, we study the feasibility of combining RBT with MBT by using the upcoming version of UML Testing Profile (UTP 2) as the mechanism. We present potential traceability between RBT and UTP 2 concepts.

JUNIPER: towards modeling approach enabling efficient platform for heterogeneous big data analysis

Big Data is a modern phenomenon that promises to bring unprecedented economical benefits. Hadoop-like MapReduce implementations has gained a well deserved popularity by providing an open-source data management solution running on commodity PC clusters and with a potential of Big Data scale. Nevertheless, there are many critical problems, for which solutions based on HPCs, FPGA-enabled nodes and providing real-time guaranties may offer a cost-efficient solution for data processing. JUNIPER is a European research projects that is carried out by an international consortium aiming at developing a Big Data analysis platform. In this article, we present an integral part of JUNIPER - a modeling approach, which helps abstracting the data processing stages and wrap the communication between them. This approach is also applied to specify the timing constraints. We illustrate our approach on a real-life application of credit card transaction processing developed by petaFuel.

The FITTEST Tool Suite for Testing Future Internet Applications

Future Internet applications are expected to be much more complex and powerful, by exploiting various dynamic capabilities For testing, this is very challenging, as it means that the range of possible behavior to test is much larger, and moreover it may at the run time change quite frequently and significantly with respect to the assumed behavior tested prior to the release of such an application. The traditional way of testing will not be able to keep up with such dynamics. The Future Internet Testing (FITTEST) project (http://crest.cs.ucl.ac.uk/fittest/), a research project funded by the European Commission (grant agreement n. 257574) from 2010 till 2013, was set to explore new testing techniques that will improve our capacity to deal with the challenges of testing Future Internet applications. Such techniques should not be seen as replacement of the traditional testing, but rather as a way to complement it. This paper gives an overview of the set of tools produced by the FITTEST project, implementing those techniques.

Designing Swarms of Cyber-Physical Systems: the H2020 CPSwarm Project: Invited Paper

Cyber-Physical Systems (CPS) find applications in a number of large-scale, safety-critical domains e.g. transportation, smart cities, etc. As a matter of fact, the increasing interactions amongst different CPS are starting to generate unpredictable behaviors and emerging properties, often leading to unforeseen and/or undesired results. Rather than being an unwanted byproduct, these interactions could, however, become an advantage if they were explicitly managed, and accounted, since the early design stages. The CPSwarm project, presented in this paper, aims at tackling these kinds of challenges by easing development and integration of complex herds of heterogeneous CPS. Thanks to CPSwarm, systems designed through a combination of existing and emerging tools, will collaborate on the basis of local policies and exhibit a collective behavior capable of solving complex, real-world, problems. Three real-world use cases will demonstrate the validity of foundational assumptions of the presented approach as well as the viability of the developed tools and methodologies.

Integration of Hawk for Model Metrics in the MEASURE Platform.

The MEASURE project aims to integrate metrics across all phases of the software development lifecycle into a single decision support platform. For the earlier phases, metrics can be derived from models. Industrial use of model-driven engineering produces large model repositories, and high-performance querying is key to keep their metrics up to date. This paper presents an integration between the MEASURE metrics platform and the Hawk model indexing tool. Hawk was improved in several ways, such as adding support for the new Modelio metamodelling framework, or allowing Hawk servers to be provisioned through configuration files rather than through its web services. MEASURE and Hawk were then combined successfully to extract metrics from Modelio models of various domains, and Hawk was able to index and efficiently answer queries about the 2GB collection of models used by Softeam to develop Modelio.