Taisy Silva Weber

FIONA: a fault injector for dependability evaluation of Java-based network applications

The use of network applications for high availability systems requires the validation of its fault tolerance mechanisms to avoid unexpected behavior during execution. FIONA is a fault injection tool to experimentally validate these mechanisms of Java distributed applications. The tool uses JVMTI, a new interface for the development of debugging and monitoring tools that enables the instrumentation of Java applications. This approach provides complete transparency between the application under test and the fault injection tool, as well as portability. FIONA injects communication faults, making it possible to conduct the dependability evaluation of UDP based network protocols developed in Java.

A kernel-based communication fault injector for dependability testing of distributed systems

Software-implemented fault injection is a powerful strategy to test fault-tolerant protocols in distributed environments. In this paper, we present ComFIRM, a communication fault injection tool we developed which minimizes the probe effect on the tested protocols. ComFIRM explores the possibility to insert code directly inside the Linux kernel in the lowest level of the protocol stack through the load of modules. The tool injects faults directly into the message exchange subsystem, allowing the definition of test scenarios from a wide fault model that can affect messages being sent and/or received. Additionally, the tool is demonstrated in an experiment which applies the fault injector to evaluate the behavior of a group membership service under communication faults.

A Network-Level Distributed Fault Injector for Experimental Validation of Dependable Distributed Systems

The use of Java for distributed systems and high-available applications demands the validation of their fault tolerance mechanisms to avoid unexpected behavior during execution. We present an extension of FIONA (fault injector oriented to network applications), a fault injection environment to experimentally validate the dependability of distributed Java applications. The main features of this extension are: its distributed architecture, which allows centralized configuration of multiple faults scenarios, and the support for a wider fault model associated with distributed systems, which includes network partitioning. For monitoring, FIONA supports the collection of log information and includes a helper application to integrate this information in a global log for post-mortem dependability analysis. FIONA is simple to operate, and we expect it to facilitate the conduction of validation experiments by application developers and testers

Applying FIRMAMENT to test the SCTP communication protocol under network faults

How to apply a fault injector to evaluate the dependability of a network protocol implementation is the main focus of this paper. In the last years, we have been developing FIRMAMENT, a tool to inject faults directly into messages that pass through the kernel protocol stack. SCTP is a promising new protocol over IP that, due its enhanced reliability, is competing with TCP where dependability has to be guaranteed. Using FIRMAMENT we evaluate the error coverage and the performance degradation of SCTP under faults. Performing a complete fault injection campaign over a third party software give us a deep insight about the additional test strategies that are needed to reach significant dependability measures.

Experimental dependability assessment using a faultload specification tool

Use of two or more fault injection tools in a test campaign enriches the scenario obtained from a test execution. Faultloads represent the main input for these tools but their specification mechanisms lack usability and expressiveness. This paper presents a full test scenario featuring the use of jFaultload, which applies Java for the specification of fault-loads and translates them to specific formats that are appropriate to each available fault injector. FIRMAMENT, a fault injector for communication systems, was integrated in the environment and completes the test scenario. The service under test used to demonstrate the usability and expressiveness of our solution is a video streaming session using RTP Protocol.

A practical methodology for experimental fault injection to test complex network-based systems

A methodology for dependability assessment of complex computer systems, such as fault tolerant grids, is presented in this paper. The methodology uses communication fault injection and was built by adapting a widely accepted approach for performance analysis. To demonstrate its applicability and usefulness, the methodology was applied to a third party grid platform using a fault injector we are developing. The paper reasons about the advantages of this methodology to perform fault injection campaigns in the prototype phase of a system.

Dependability evaluation of distributed systems through partitioning fault injection

Fault injection intents to insert artificial faults in a target system under test to assess their behavior in abnormal situations. However, few efforts focus on partitioning faults that affect distributed applications. The difficulties in reproducing network partitionings remain open. This paper presents a novel tool that emulate partitioning in a real IP network to evaluate the dependability of distributed applications. The tool is composed of distributed instances of a communication fault injector and an experiment coordinator. Using the tool, partitioning scenarios can be described at a high level of abstraction allowing to create high expressive faultloads and realize effective fault injection experiments.

Injecting communication faults to experimentally validate java distributed applications

The use of Java to develop highly available network services and the inherent unreliability of communication systems require the validation of its fault tolerance mechanisms to avoid unexpected behavior during execution. Tools for testing distributed programs that must remain operational in a faulty environment are not common and are generally difficult to operate. One possible solution is applying a fault injector to emulate communication faults and test the behavior of the distributed application under these faults. We developed Jaca.net, an extension of the fault injector based on computational reflection Jaca, to include UDP communication faults to its fault model. Potential target applications of Jaca and Jaca.net are the ones developed in Java. Using Jaca.net, software developers can complete the test cycle validating the high level fault detection and recovery mechanisms built on their applications.

Practical experience designing and debugging an FPGA for a real-time ethernet industrial bus

Several debugging methods are available for FPGA design, including logical simulation and physical debug. We face the challenge to systematize and apply a simple methodology using these methods to develop a communication processing module for a distributed control system with real time constraints. This methodology must follow some restrictions as to be easy to learn, use only available tools and not conflict with techniques employed by our industrial partner. The hardware module must be designed from scratch and must replace commercial communication modules in new devices. It will be responsible for real time interactions between programmable controllers and remote IO in large distributed control systems. The paper describes our experience designing the processing module, the methodology applied and an evaluation of this methodology during the project development.

A Software Fault Injector to Validate Implementations of a Safety Communication Protocol

Communication faults can lead to irreparable damages. Faulty messages increase the risk of accidents in distributed industrial environments. When the risk is high, safety standards force communication over safety protocols. For each new controller, input and output device, a new implementation of a safety protocol is required. Its validation must strictly follow safety standards, which imposes fault injection to test the protocol. This paper introduces a communication fault injector to validate implementations of PROFIsafe, one of the safety protocols standardized by IEC 61784-3. To meet low cost and minor intrusiveness requirements, the injector was developed in software and uses an efficient kernel module, PF-RING, to capture messages directly from network interfaces. A complete functional version of the injector is ready to use and was applied to validate a prototype implementation of PROFIsafe.