Martin Hiller

Executable assertions for detecting data errors in embedded control systems

In order to be able to tolerate the effects of faults, we must first detect the symptoms of faults, i.e. the errors. This paper evaluates the error detection properties of an error detection scheme based on the concept of executable assertions aiming to detect data errors in internal signals. The mechanisms are evaluated using error injection experiments in an embedded control system. The results show that using the mechanisms allows one to obtain a fairly high detection probability for errors in the areas monitored by the mechanisms. The overall detection probability for errors injected to the monitored signals was 74%, and if only errors causing failure are taken into account we have a detection probability of over 99%. When subjecting the target system to random error injections in the memory areas of the application, i.e., not only the monitored signals, the detection probability for errors that cause failure was 81%.

An approach for analysing the propagation of data errors in software

We present a novel approach for analysing the propagation of data errors in software. The concept of error permeability is introduced as a basic measure upon which we define a set of related measures. These measures guide us in the process of analysing the vulnerability of software to find the modules that are most likely exposed to propagating errors. Based on the analysis performed with error permeability and its related measures, we describe how to select suitable locations for error detection mechanisms (EDMs) and error recovery mechanisms (ERMs). A method for experimental estimation of error permeability, based on fault injection, is described and the software of a real embedded control system analysed to show the type of results obtainable by the analysis framework. The results show that the developed framework is very useful for analysing error propagation and software vulnerability and for deciding where to place EDMs and ERMs.

PROPANE: an environment for examining the propagation of errors in software

In order to produce reliable software, it is important to have knowledge on how faults and errors may affect the software. In particular, designing efficient error detection mechanisms requires not only knowledge on which types of errors to detect but also the effect these errors may have on the software as well as how they propagate through the software. This paper presents the Propagation Analysis Environment (PROPANE) which is a tool for profiling and conducting fault injection experiments on software running on desktop computers. PROPANE supports the injection of both software faults (by mutation of source code) and data errors (by manipulating variable and memory contents). PROPANE supports various error types out-of-the-box and has support for user-defined error types. For logging, probes are provided for charting the values of variables and memory areas as well as for registering events during execution of the system under test. PROPANE has a flexible design making it useful for development of a wide range of software systems, e.g., embedded software, generic software components, or user-level desktop applications. We show examples of results obtained using PROPANE and how these can guide software developers to where software error detection and recovery could increase the reliability of the software system.

On the placement of software mechanisms for detection of data errors

An important aspect in the development of dependable software is to decide where to locate mechanisms for efficient error detection and recovery. We present a comparison between two methods for selecting locations for error detection mechanisms, in this case executable assertions (EAs), in black-box, modular software. Our results show that by placing EAs based on error propagation analysis one may reduce the memory and execution time requirements as compared to experience- and heuristic-based placement while maintaining the obtained detection coverage. Further, we show the sensitivity of the EA-provided coverage estimation on the choice of the underlying error model. Subsequently, we extend the analysis framework such that error-model effects are also addressed and introduce measures for classifying signals according to their effect on system output when errors are present. The extended framework facilitates profiling of software systems from varied dependability perspectives and is also less susceptible to the effects of having different error models for estimating detection coverage.

An experimental comparison of fault and error injection

The complex interactions between faults, errors, failures and fault handling mechanisms can be studied via injection experiments. This paper presents an investigation of both fault and error injection techniques for emulating software faults. For evaluation, 1600 software faults and 5400 time-triggered errors were injected into an embedded real-time system. The cost-related results are: (i) the time required to create a fault set for fault injection was about 120 times longer than the time required to create an error set for time-triggered injection, and (ii) the execution time for the time-triggered error injection experiments was four times shorter than for the fault injection experiments. However, the error injection would be only 1.3 times faster if another strategy for fault injection had been used. Furthermore, failure symptom-related results are: (i) the test case had a greater influence than the fault type on the failure symptoms for fault injections, (ii) the error type had a greater influence on the failure symptoms for time-triggered error injections than had the test case, and (iii) the error type had a larger impact on the failure symptoms than the fault type.

Assessing inter-modular error propagation in distributed software

With the functionality of most embedded systems based on software (SW), interactions amongst SW modules arise, resulting in error propagation across them. During SW development, it would be helpful to have a framework that clearly demonstrates the error propagation and containment capabilities of the different SW components. In this paper, we assess the impact of inter-modular error propagation. Adopting a white-box SW approach, we make the following contributions: (a) we study and characterize the error propagation process and derive a set of metrics that quantitatively represents the inter-modular SW interactions, (b) we use a real embedded target system used in an aircraft arrestment system to perform fault-injection experiments to obtain experimental values for the metrics proposed, (c) we show how the set of metrics can be used to obtain the required analytical framework for error propagation analysis. We find that the derived analytical framework establishes a very close correlation between the analytical and experimental values obtained. The intent is to use this framework to be able to systematically develop SW such that inter-modular error propagation is reduced by design.

Component-Based Synthesis of Dependable Embedded Software

Standardized and reusable software (SW) objects (or SW components - in-house or pre-fabricated) are increasingly being used to reduce the cost of software (SW) development. Given that the basic components may not have been developed with dependability as primary driver, these components need to be adapted to deal with errors from their environment. To achieve this, error containment wrappers are added to increase the reliability of the components. In this paper, we first present a modular specification approach using fault intolerant components, based on the concepts of category theory. We further introduce the concept of wrapper consistency, based upon which, we present an algorithm that systematically generates globally consistent fault containment wrappers for each component, to make them fault tolerant. Subsequently, we enhance the initial modular specification to deal with the wrapped components, and show that safety properties of the system are preserved under composition only if the wrappers are globally consistent.

On systematic design of globally consistent executable assertions in embedded software

Over the design of software (SW) used in provisioning of dependable services, Executable Assertions (EAs) are seeing increasing usage in aiding detection of data errors. Given the requirements for provision of service despite faults, early detection of system states that can potentially lead to system failure is valuable. We address the issue of ascertaining whether localized EAs in individual modules add up complementarily to implement a global EA/property. We first show that detection of globally compliant EAs is NP-complete. Thus, we develop a two-pass approach for our objective. In the first pass, we introduce the consistency property of EAs and use it to ascertain global conformity across all EAs. The second pass, analogous to predicate transformers, generates globally consistent EAs when any inconsistency is flagged in the first pass. We show the applicability of our approach on a real embedded system. Initial results obtained show that our framework is able to detect inherent vulnerabilities (due to placement of mismatched EAs) that were previously undetected. Our intent is automation of this approach, which can be incorporated in a compiler.

An approach to specify and test component-based dependable software

Components (in-house or pre-fabricated) are increasingly being used to reduce the cost of software development. Given that these components may not have not been developed with dependability as a driver, the components need to be adapted to deal with errors coming from their environment. To achieve this, error containment wrappers are often added to increase the robustness of such components. Adopting a gray-box perspective of software, we first present a modular approach for specifying and verifying embedded software made from components, based on concepts from category theory. This modular approach allows the system designer to check for semantic compatibility. To generate the error containment wrappers needed for adaptation, we subsequently present an algorithm that systematically generates the required wrappers. Using the information obtained through wrapper design, we develop an approach to identify relevant test cases to test individual components. We further exploit the modularity of the specification to identify the relevant test cases to perform testing at different levels of SW abstraction.

A control theory approach for analyzing the effects of data errors in safety-critical control systems

Computers are increasingly used for implementing control algorithms in safety-critical embedded applications, such as engine control, braking control and flight surface control. Addressing the consequent coupling of control performance with computer related errors, this paper develops a composite computer dependability/control theory methodology for analyzing the effects data errors have on control system dependability. The effect is measured as the resulting control error (defined as the difference between the desired value of a physical properly and its actual value). We use maximum bounds on this measure as the criterion for control system failure (i.e., if the control error exceeds a certain threshold, the system has failed). In this paper we a) present suitable models of computer faults for analysis of control level effects and related analysis methods, and b) apply traditional control theory analysis methods for understanding the effects of data errors on system dependability An automobile slip-control brake-system is used as an example showing the viability of our approach.