Mario Zenha-Rela

Test Case Evaluation and Input Domain Reduction strategies for the Evolutionary Testing of Object-Oriented software

In Evolutionary Testing, meta-heuristic search techniques are used for generating test data. The focus of our research is on employing evolutionary algorithms for the structural unit-testing of Object-Oriented programs. Relevant contributions include the introduction of novel methodologies for automation, search guidance and Input Domain Reduction; the strategies proposed were empirically evaluated with encouraging results. Test cases are evolved using the Strongly-Typed Genetic Programming technique. Test data quality evaluation includes instrumenting the test object, executing it with the generated test cases, and tracing the structures traversed in order to derive coverage metrics. The methodology for efficiently guiding the search process towards achieving full structural coverage involves favouring test cases that exercise problematic structures. Purity Analysis is employed as a systematic strategy for reducing the search space.

Work in Progress: Self Evaluation Through Monitored Peer Review Using the Moodle Platform

We developed a module for the Moodle open-source eLearning platform through which undergraduate students can Build, Answer and Evaluate by blind peer review questions related to a course subject. In the first learning phase the students can build and submit questions. Selected questions are candidates for inclusion in later tests. After the test a reference solution and evaluation criteria is made public. The students answers are then randomly distributed so that each student has to evaluate at least three of his peers tests using double blind peer review. After all tests are cross-evaluated by the community of students, the system grades them by majority vote in each question. The need to: i) build meaningful questions; ii) present a correct answer; iii) state the evaluation criteria; iv) answer to subject-related questions; and finally v) evaluate; and vi) justify a grade attributed to a classmate, is a powerful community-aware context in which students develop competences on the subjects topic while simultaneously have to reason about their learning process

An evolutionary approach for performing structural unit-testing on third-party object-oriented Java software

Evolutionary Testing is an emerging methodology for automatically generating high quality test data. The focus of this paper is on presenting an approach for generating test cases for the unit-testing of object-oriented programs, with basis on the information provided by the structural analysis and interpretation of Java bytecode and on the dynamic execution of the instrumented test object. The rationale for working at the bytecode level is that even when the source code is unavailable, insight can still be obtained and used to guide the search-based test case generation process. Test cases are represented using the Strongly Typed Genetic Programming paradigm, which effectively mimics the polymorphic relationships, inheritance dependences and method argument constraints of object-oriented programs.

Automated Reliability Prediction from Formal Architectural Descriptions

Quantitative assessment of quality attributes (i.e., non-functional requirements, such as performance, safety or reliability) of software architectures during design supports important early decisions and validates the quality requirements established by the stakeholder. In current practice, these quality requirements are most often manually checked, which is time-consuming and error-prone due to the overwhelmingly complex designs. We propose an automated approach to assess the reliability of software architectures. It consists in extracting a Markov model from the system specification written in an Architecture Description Language (ADL). Our approach translates the specified architecture to a high-level probabilistic model-checking language, supporting system validation and quantitative reliability prediction against usage profile, component arrangement and architectural styles. We validate our approach by applying it to different architectural styles and comparing those with two different quantitative reliability assessment methods presented in the literature: the composite and the hierarchical methods.

Enabling object reuse on genetic programming-based approaches to object-oriented evolutionary testing

Recent research on search-based test data generation for Object-Oriented software has relied heavily on typed Genetic Programming for representing and evolving test data. However, standard typed Genetic Programming approaches do not allow Object Reuse; this paper proposes a novel methodology to overcome this limitation. Object Reuse means that one instance can be passed to multiple methods as an argument, or multiple times to the same method as arguments. In the context of Object-Oriented Evolutionary Testing, it enables the generation of test programs that exercise structures of the software under test that would not be reachable otherwise. Additionally, the experimental studies performed show that the proposed methodology is able to effectively increase the performance of the test data generation process.

Strongly-typed genetic programming and purity analysis: input domain reduction for evolutionary testing problems

Search-based test case generation for object-oriented software is hindered by the size of the search space, which encompasses the arguments to the implicit and explicit parameters of the test objects public methods. The performance of this type of search problems can be enhanced by the definition of adequate Input Domain Reduction strategies. The focus of our on-going work is on employing evolutionary algorithms for generating test data for the structural unit-testing of Java programs. Test cases are represented and evolved using the Strongly-Typed Genetic Programming paradigm; Purity Analysis is particularly useful in this situation because it provides a means to automatically identify and remove Function Set entries that do not contribute to the definition of interesting test scenarios.

FIRED -- Fault Injector for Reconfigurable Embedded Devices

Reconfigurable embedded devices built on SRAM-based Field Programmable Gate Arrays (FPGA) are being increasingly used in critical embedded applications. However, the susceptibility of such memory cells to Single Event Upsets (SEU) requires the use of fault tolerant designs, for which fault injection is still the most accepted verification technique. This paper describes FIRED, a fault injector targeted at SRAM-based FPGAs for dependability evaluation of critical systems. This tool is able to perform hardware fault injection in real-time, by inserting bitflips at the SRAM cells through partial dynamic reconfiguration. These faults may produce errors in the design of the VHDL or Verilog modules deployed in the FPGA. A case study of a fault injection campaign in a PID-based cruise control system is used to evaluate the capabilities of FIRED, namely its capacity of injecting faults while a physical application is being controlled.

Improving self-adaptation planning through software architecture-based stochastic modeling

We propose a formal automated approach to translate from an ADL to a DTMC.We address issues of today self-adaptive systems.We assessed dynamically the impact of each strategy in the system quality.Our approach presents better results than traditional planning algorithms.Our approach presents good scalability and performance results. The ever-growing complexity of software systems makes it increasingly challenging to foresee at design time all interactions between a system and its environment.Most self-adaptive systems trigger adaptations through operators that are statically configured for specific environment and system conditions. However, in the occurrence of uncertain conditions, self-adaptive decisions may not be effective and might lead to a disruption of the desired non-functional attributes.To address this, we propose an approach that improves the planning stage by predicting the outcome of each strategy. In detail, we automatically derive a stochastic model from a formal architecture description of the managed system with the changes imposed by each strategy. Such information is used to optimize the self-adaptation decisions to fulfill the desired quality goals.To assess the effectiveness of our approach we apply it to a cloud-based news system and predicted the reliability for each possible adaptation strategy. The results obtained from our approach are compared to a representative static planning algorithm as well as to an oracle that always makes the ideal decision. Experiments show that our method improves both availability and cost when compared to the static planning algorithm, while being close to the oracle.Our approach may therefore be used to optimize self-adaptation planning.

FPGA-based weblab infrastructures guidelines and a prototype implementation example

Recent trends show an increasing number of weblabs, implemented at universities and schools, supporting practical training in technical courses and providing the ability to remotely conduct experiments. However, their implementation is typically based on individual architectures, unable of being reconfigured with different instruments/modules usually required by every experiment. In this paper, we discuss practical guidelines for implementing reconfigurable weblabs that support both local and remote control interfaces. The underlying infrastructure is based on reconfigurable, low-cost, FPGA-based boards supporting several peripherals that are used for the local interface. The remote interface is powered by a module capable of communicating with an Ethernet based network and that can either correspond to an internal core of the FPGA or an external device. These two approaches are discussed in the paper, followed by a practical implementation example.

Reliability Analysis of Software Architecture Evolution

Software engineers and practitioners regard software architecture as an important artifact, providing the means to model the structure and behavior of systems and to support early decisions on dependability and other quality attributes. Since systems are most often subject to evolution, the software architecture can be used as an early indicator on the impact of the planned evolution on quality attributes. We propose an automated approach to evaluate the impact on reliability of architecture evolution. Our approach provides relevant information for architects to predict the impact of component reliabilities, usage profile and system structure on the overall reliability. We translate a systems architectural description written in an Architecture Description Language (ADL) to a stochastic model suitable for performing a thorough analysis on the possible architectural modifications. We applied our method to a case study widely used in research in which we identified the reliability bottlenecks and performed structural modifications to obtain an improved architecture regarding its reliability.