Jonny Vinter

GOOFI: generic object-oriented fault injection tool

We present a new fault injection tool called GOOFI (Generic Object-Oriented Fault Injection). GOOFI is designed to be adaptable to various target systems and different fault injection techniques. The tool is highly portable between different host platforms since it relies on the Java programming language and an SQL compatible database. The current version of the tool supports pre-runtime software implemented fault injection and scan-chain implemented fault injection.

MODIFI: a MODel-implemented fault injection tool

Fault injection is traditionally divided into simulation-based and physical techniques depending on whether faults are injected into hardware models, or into an actual physical system or prototype. Another classification is based on how fault injection mechanisms are implemented. Well known techniques are hardware-implemented fault injection (HIFI) and softwareimplemented fault injection (SWIFI). For safety analyses during model-based development, fault injection mechanisms can be added directly into models of hardware, models of software or models of systems. This approach is denoted by the authors as model-implemented fault injection. This paper presents the MODIFI (MODel-Implemented Fault Injection) tool. The tool is currently targeting behaviour models in Simulink. Fault models used by MODIFI are defined using XML according to a specific schema file and the fault injection algorithm uses the concept of minimal cut sets (MCS) generation. First, a user defined set of single faults are injected to see if the system is tolerant against single faults. Single faults leading to a failure, i.e. a safety requirement violation, are stored in a MCS list together with the corresponding counterexample. These faults are also removed from the fault space used for subsequent experiments. When all single faults have been injected, the effects of multiple faults are investigated, i.e. two or more faults are introduced at the same time. The complete list of MCS is finally used to automatically generate test cases for efficient fault injection on the target system.

Assembly-Level pre-injection analysis for improving fault injection efficiency

This paper describes a fully automated pre-injection analysis technique aimed at reducing the cost of fault injection campaigns. The technique optimizes the fault-space by utilizing assembly-level knowledge of the target system in order to place single bit-flips in registers and memory locations only immediately before these are read by the executed instructions. This way, faults (time-location pairs) that are overwritten or have identical impact on program execution are removed. Experimental results obtained by random sampling of the optimized fault-space and the complete (non-optimized) fault-space are compared for two different workloads running on a MPC565 microcontroller. The pre-injection analysis yields an increase of one order of magnitude in the effectiveness of faults, a reduction of the fault-space of two orders of magnitude in the case of CPU-registers and four to five orders of magnitude in the case of memory locations, while preserving a similar estimation of the error detection coverage.

Reducing critical failures for control algorithms using executable assertions and best effort recovery

Systems that use f+1 computer nodes to tolerate f node failures ordinarily require that the computer nodes have strong failure semantics, i.e. a node should either produce correct results or no results at all. We show that this requirement can be relaxed for control applications, as control algorithms inherently compensate for a class of value failures. Value failures occur when an error escapes the error detection mechanisms in the computer node and an erroneous value is sent to the actuators of the control system. Fault injection experiments show that 89% of the value failures caused by bit flips in a CPU had no or minor impact on the controlled object. However, the experiments also show that 11% of the value failures had severe consequences. These failures were caused by bit flips affecting the state variables of the control algorithm. Another set of fault injection experiments showed that the percentage of value failures with severe consequences was reduced to 3% when the state variables were protected with executable assertions and best-effort recovery mechanisms.

Experimental evaluation of time-redundant execution for a brake-by-wire application

This paper presents an experimental evaluation of a brake-by-wire application that tolerates transient faults by temporal error masking. A specially designed real-time kernel that masks errors by triple time-redundant execution and voting executes the application on a fail-stop computer node. The objective is to reduce the number of node failures by masking errors at the computer node level. The real-time kernel always executes the application twice to detect errors, and ensures that a fail-stop failure occurs if there is not enough CPU-time available for a third execution and voting. Fault injection experiments show that temporal error masking reduced the number of fail-stop failures by 42% compared to executing the brake-by-wire task without time redundancy.

Driver performance in the presence of adaptive cruise control related failures: Implications for safety analysis and fault tolerance

This study explored how failures related to an adaptive cruise control (ACC) were handled by drivers and what the effects on safety can be. The experimental study included forty-eight subjects and was performed in a moving base driving simulator equipped with an ACC. Each subject experienced two different failures in separate scenarios. In total, the study included four different failures, i.e., Unwanted acceleration, Complete lack of deceleration, Partial lack of deceleration, and Speed limit violation. The outcome of each failure scenario has been categorized based on whether the driver managed to avoid a collision or not. For the outcomes where collisions were successfully avoided, the situations were analyzed in more detail and classified according to the strategy used by the driver. Besides showing that partial lack of deceleration caused more collisions than complete lack of deceleration (43% compared to 14% of the participants colliding), the results also indicate a preference among drivers to steer and change lane rather than to apply the brakes when faced with acceleration and deceleration failures. A trade off relationship was identified between allowing a failing ACC to stay operational and on the other hand disabling it when an error is detected. Keeping the system operational can cause confusion about the mode of the system but as the results of the study indicate it can also improve the situation by reducing impact speed.

Experimental dependability evaluation of a fail-bounded jet engine control system for unmanned aerial vehicles

This paper presents an experimental evaluation of a prototype jet engine controller intended for unmanned aerial vehicles (UAVs). The controller is implemented with commercial off-the-shelf (COTS) hardware based on the Motorola MPC565 microcontroller. We investigate the impact of single event upsets (SEUs) by injecting single bit-flip faults into main memory and CPU registers via the Nexus on-chip debug interface of the MPC565. To avoid the injection of non-effective faults, automated pre-injection analysis of the assembly code was utilized. Due to the inherent robustness of the software, most injected faults were still non-effective (69.4%) or caused bounded failures having only minor effect on the jet engine (7.0%), while 20.1% of the errors were detected by hardware exceptions and 1.9% were detected by executable assertions in the software. The remaining 1.6% is classified as critical failures. A majority of the critical failures were caused by erroneous Booleans or type conversions involving Booleans.

On the design of robust integrators for fail-bounded control systems

This paper describes the design and evaluation of a robust integrator for software-implemented control systems. The integrator is constructed as a generic component in the Simulink design tool, and can thus be used for robust implementation of a wide range of control algorithms. The integrator is designed to support the failbounded failure model for transient bit-flips that may occur in the CPU, main memory and I/O circuits of a control system. In particular, it allows the control system to detect and recover from bit-flips that cause data errors. Robustness is achieved by sequentially executing duplicated integrator code on the same processor to support error detection, and through the use of a recovery buffer that allows a roll-back to the previous integrator state when an error is detected. The effectiveness of the robust integrator was evaluated through fault injection experiments with a PI controller, where single bit flips were injected inside the CPU of the control system. No violations of the fail-bounded model were observed in the experiments.

A Tunable Add-On Diagnostic Protocol for Time-Triggered Systems

We present a tunable diagnostic protocol for generic time-triggered (TT) systems to detect crash and send/receive omission faults. Compared to existing diagnostic and membership protocols for TT systems, it does not rely on the single-fault assumption and tolerates malicious faults. It runs at the application level and can be added on top of any TT system (possibly as a middleware component) without requiring modifications at the system level. The information on detected faults is accumulated using a penalty/reward algorithm to handle transient faults. After a fault is detected, the likelihood of node isolation can be adapted to different system configurations, including those where functions with different criticality levels are integrated. Using actual automotive and aerospace parameters, we experimentally demonstrate the transient fault handling capabilities of the protocol.

Modular certification support — the DECOS concept of generic safety cases

The integrated EU-project DECOS (dependable embedded components and systems) developed an integrated architecture for safety critical embedded systems. To reduce the effort for the certification of DECOS based applications it provides support for modular certification based on generic safety cases. This means that a safety case of a DECOS based application only contains the application-specific issues and reuses the safety arguments of the generic safety cases of the DECOS platform. The concept of safety cases was complemented by trust cases which tackle further aspects such as security.