Behrooz Sangchoolie

A Study of the Impact of Single Bit-Flip and Double Bit-Flip Errors on Program Execution

This paper presents the results of an extensive experimental study of bit-flip errors in instruction set architecture registers and main memory locations. Comprising more than two million fault injection experiments conducted with thirteen benchmark programs, the study provides insights on whether it is necessary to consider double bit-flip errors in dependability benchmarking experiments. The results show that the proportion of silent data corruptions in the program output, is almost the same for single and double bit errors. In addition, we present detailed statistics about the error sensitivity of different target registers and memory locations, including bit positions within registers and memory words. These show that the error sensitivity varies significantly between different bit positions and registers. An important observation is that injections in certain bit positions always have the same impact regardless of when the error is injected.

A Study of the Impact of Bit-Flip Errors on Programs Compiled with Different Optimization Levels

In this paper we study the impact of compiler optimizations on the error sensitivity of twelve benchmark programs. We conducted extensive fault injection experiments where bit-flip errors were injected in instruction set architecture registers and main memory locations. The results show that the percentage of silent data corruptions (SDCs) in the output of the optimized programs is only marginally higher compare to that observed for the non-optimized programs. This suggests that compiler optimizations can be used in safety- and mission-critical systems without increasing the risk that the system produces undetected erroneous outputs. In addition, we investigate to what extent the source code implementation of a program affects the error sensitivity of a program. To this end, we perform experiments with five implementations of a bit count algorithm. In this investigation, we consider the impact of the implementation as well as compiler optimizations. The results of these experiments give valuable insights into how compiler optimizations can be used to reduce error sensitive of registers and main memory sections. They also show how sensitive locations requiring additional protection, e.g., by the use of software-based fault tolerance techniques, can be identified.

On the impact of hardware faults --- an investigation of the relationship between workload inputs and failure mode distributions

Technology scaling of integrated circuits is making transistors increasingly sensitive to process variations, wear-out effects and ionizing particles. This may lead to an increasing rate of transient and intermittent errors in future microprocessors. In order to assess the risk such errors pose to safety critical systems, it is essential to investigate how temporary errors in the instruction set architecture (ISA) registers and main memory locations influence the behaviour of executing programs. To this end, we investigate --- by means of extensive fault injection experiments --- how such errors affect the execution of four target programs. The paper makes three contributions. First, we investigate how the failure modes of the target programs vary for different input sets. Second, we evaluate the error coverage of a software-implemented hardware fault tolerant technique that relies on triple-time redundant execution, majority voting and forward recovery. Third, we propose an approach based on assembly language metrics which can be used to correlate the dynamic fault-free behaviour of a program with its failure mode distribution obtained by fault injection.

A Comparison of Inject-on-Read and Inject-on-Write in ISA-Level Fault Injection

ISA-level fault injection, i.e. the injection of bit-flip faults in Instruction Set Architecture (ISA) registers and main memory words, is widely used for studying the impact of transient and intermittent hardware faults in computer systems. This paper compares two techniques for ISA-level fault injection: inject-on-read, and inject-on-write. The first technique injects bit-flips in a data-item (the content of a register or memory word) just before the data-item is read by a machine instruction, while the second one injects bit-flips in a data-item just after it has been updated by a machine instruction. In addition, the paper compares two variants of inject-on-read, one where all faults are given the same weight and one where weight factors are used to reflect the time a data-item spends in a register or memory word. The weighted injected-on-read aims to accurately model soft errors that occur when an ionizing particle perturbs a data-item while it resides in an ISA register or a memory word. This is in contrast to inject-on-write, which emulates errors that propagate into an ISA register or a memory word. Our experiments show significant differences in the results obtained with the three techniques.

Light-Weight Techniques for Improving the Controllability and Efficiency of ISA-Level Fault Injection Tools

ISA-level fault injection, i.e. the injection of bitflip faults in Instruction Set Architecture (ISA) registers and main memory words, is widely used for studying the impact of transient and intermittent hardware faults. ISA-level fault injection tools can be characterized by different properties such as repeatability, observability, reachability, intrusiveness, efficiency and controllability. This paper presents two preinjection analysis techniques that improve controllability and efficiency using object code analysis. To improve controllability, we propose a technique for identifying the type of data that is stored in a potential target location. This allows the user to selectively direct fault injections to addresses, data and/or control information. Experimental results show that the data type of 84-100% of the targets locations in 8 programs were successfully identified by this technique. The second technique improves efficiency by fault pruning, i.e., by avoiding injection of faults that is known a priori to be detected by the tested system. This technique leverage the fact that faults in certain bits in the program counter and the stack pointer are always detected by machine exceptions. We show that exclusion of these bits from the fault space could significantly prune the fault space and reduce the time it takes to conduct a fault injection campaign.

Back-to-Back Fault Injection Testing in Model-Based Development

Today, embedded systems across industrial domains e.g., avionics, automotive are representatives of software-intensive systems with increasing reliance on software and growing complexity. It has become critically important to verify software in a time, resource and cost effective manner. Furthermore, industrial domains are striving to comply with the requirements of relevant safety standards. This paper proposes a novel workflow along with tool support to evaluate robustness of software in model-based development environment, assuming different abstraction levels of representing software. We then show the effectiveness of our technique, on a brake-by-wire application, by performing back-to-back fault injection testing between two different abstraction levels using MODIFI for the Simulink model and GOOFI-2 for the generated code running on the target microcontroller. Our proposed method and tool support facilitates not only verifying software during early phases of the development lifecycle but also fulfilling back-to-back testing requirements of ISO 26262 [1] when using model-based development.

Towards Benchmarking of Functional Safety in the Automotive Industry

Functional safety is becoming increasingly important in the automotive industry to deal with the growing reliance on the electrical and/or electronic (E/E) systems and the associated complexities. The introduction of ISO 26262, a new standard for functional safety in road vehicles, has made it even more important to adopt a systematic approach of evaluating functional safety. However, standard assessment methods of benchmarking functional safety of automotive systems are not available as of today. This is where the BeSafe (Benchmarking of Functional Safety) project comes into the picture. BeSafe project aims to lay the foundation for benchmarking functional safety of automotive E/E systems. In this paper, we present a brief overview of the project along with the benchmark targets that we have identified as relevant for the automotive industry, assuming three abstraction layers (model, software, hardware). We then define and discuss a set of benchmark measures. Next, we propose a benchmark framework encompassing fault/error models, methods and the required tool support. This paper primarily focuses on functional safety benchmarking from the Safety Element out of Context (SEooC) viewpoint. Finally, we present some preliminary results and highlight potential future works.

Assessing the quality of packet-level traces collected on internet backbone links

The quality of captured traffic plays an important role for decisions made by systems like intrusion detection/prevention systems (IDS/IPS) and firewalls. As these systems monitor network traffic to find malicious activities, a missing packet might lead to an incorrect decision. In this paper, we analyze the quality of packet-level traces collected on Internet backbone links using different generations of DAG cards. This is accomplished by inferring dropped packets introduced by the data collection system with help of the intrinsic structural properties inherently provided by TCP traffic flows. We employ two metrics which we believe can detect all kinds of missing packets: i) packets with ACK numbers greater than the expected ACK, indicating that the communicating parties acknowledge a packet not present in the trace; and ii) packets with data beyond the receivers window size, which with a high probability, indicates that the packet advertising the correct window size was not recorded. These heuristics have been applied to three large datasets collected with different hardware and in different environments. We also introduce flowstat, a tool developed for this purpose which is capable of analyzing both captured traces and real-time traffic. After assessing more than 400 traces (75M bidirectional flows), we conclude that at least 0.08% of the flows have missing packets, a surprisingly large number that can affect the quality of analysis performed by firewalls and intrusion detection/prevention systems. The paper concludes with an investigation and discussion of the spatial and temporal aspects of the experienced packet losses and possible reasons behind missing data in traces.