Robyn R. Lutz

Analyzing software requirements errors in safety-critical, embedded systems

The root causes of safety-related software errors in safety-critical embedded systems are analyzed. The results show that software errors identified as potentially hazardous to the system tend to be produced by different error mechanisms than those that produce nonsafety-related software errors. Safety-related software errors are shown to arise most commonly from: discrepancies between the documented requirements specifications and the requirements needed for correct functioning of the system; and misunderstandings of the interface of the software with the rest of the system. These results are used to identify methods by which requirements errors can be prevented. The goal is to reduce safety-related software errors and to enhance the safety of complex, embedded systems.<<ETX>>

Experiences using lightweight formal methods for requirements modeling

The paper describes three case studies in the lightweight application of formal methods to requirements modeling for spacecraft fault protection systems. The case studies differ from previously reported applications of formal methods in that formal methods were applied very early in the requirements engineering process to validate the evolving requirements. The results were fed back into the projects to improve the informal specifications. For each case study, we describe what methods were applied, how they were applied, how much effort was involved, and what the findings were. In all three cases, formal methods enhanced the existing verification and validation processes by testing key properties of the evolving requirements and helping to identify weaknesses. We conclude that the benefits gained from early modeling of unstable requirements more than outweigh the effort needed to maintain multiple representations.

Software engineering for safety: a roadmap

gineering for safety and proposes some directions for needed work that appears to be achievable in the near

A Software Fault Tree Approach to Requirements Analysis of an Intrusion Detection System

Requirements analysis for an intrusion detection system (IDS) involves deriving requirements for the IDS from analysis of the intrusion domain. When the IDS is, as here, a collection of mobile agents that detect, classify, and correlate system and network activities, the derived requirements include what activities the agent software should monitor, what intrusion characteristics the agents should correlate, where the IDS agents should be placed to feasibly detect the intrusions, and what countermeasures the software should initiate. This paper describes the use of software fault trees for requirements identification and analysis in an IDS. Intrusions are divided into seven stages (following Ruiu), and a fault subtree is developed to model each of the seven stages (reconnaissance, penetration, etc.). Two examples are provided. This approach was found to support requirements evolution (as new intrusions were identified), incremental development of the IDS, and prioritisation of countermeasures.

Targeting safety-related errors during software requirements analysis

This paper provides a Safety Checklist for use during the analysis of software requirements for spacecraft and others safety-critical, embedded systems. The checklist specifically targets the two most common causes of safety-related software errors: (1) inadequate interface requirements and (2) discrepancies between the documented requirements and the requirements actually needed for correct functioning of the system. The analysis criteria represented in the checklist are evaluated by application to two spacecraft projects. Use of the checklist to enhance the software-requirements analysis is shown to reduce the number of safety-related software errors.

Requirements analysis using forward and backward search

The requirements analysis of critical software components often involves a search for hazardous states and failure modes. This paper describes the integration of a forward search for consequences of reaching these forbidden modes with a backward search for contributing causes. Results are reported from two projects in which the integrated search method was used to analyze the requirements of critical spacecraft software. The search process was found to be successful in identifying some ambiguous, inconsistent, and missing requirements. More importantly, it identified four significant, unresolved requirements issues involving complex system interfaces and unanticipated dependencies. The results suggest that recent efforts by researchers to integrate forward and backward search have merit.

Empirical analysis of safety-critical anomalies during operations

Analysis of anomalies that occur during operations is an important means of improving the quality of current and future software. Although the benefits of anomaly analysis of operational software are widely recognized, there has been relatively little research on anomaly analysis of safety-critical systems. In particular, patterns of software anomaly data for operational, safety-critical systems are not well understood. We present the results of a pilot study using orthogonal defect classification (ODC) to analyze nearly two hundred such anomalies on seven spacecraft systems. These data show several unexpected classification patterns such as the causal role of difficulties accessing or delivering data, of hardware degradation, and of rare events. The anomalies often revealed latent software requirements that were essential for robust, correct operation of the system. The anomalies also caused changes to documentation and to operational procedures to prevent the same anomalous situations from recurring. Feedback from operational anomaly reports helped measure the accuracy of assumptions about operational profiles, identified unexpected dependencies among embedded software and their systems and environment, and indicated needed improvements to the software, the development process, and the operational procedures. The results indicate that, for long-lived, critical systems, analysis of the most severe anomalies can be a useful mechanism both for maintaining safer, deployed systems and for building safer, similar systems in the future.

Integrating Product-Line Fault Tree Analysis into AADL Models

Fault tree analysis (FTA) is a safety-analysis technique that has been extended recently to accommodate product-line engineering. This paper describes a tool-supported approach for integrating product-line FTA with the AADL (architecture analysis and design language) models and associated AADL Error Models for a product line. The AADL plug-in we have developed provides some automatic pruning and adaptation of the fault tree for a specific product from the product-line FTA. This work supports consistent reuse of the FTA across the systems in the product line and reduces the effort of maintaining traceability between the safety analysis and the architectural models. Incorporating the product-line FTA into the AADL models also allows derivation of basic quantitative and cut set analyses for each product-line member to help identify and eliminate design weaknesses. The tool-supported capabilities enable comparisons among candidate new members to assist in design decisions regarding redundancy, safety features, and the evaluation of alternative designs. Results from a small case study illustrate the approach.

Operational anomalies as a cause of safety-critical requirements evolution

This paper reports the results of a small study of requirements changes to the onboard software of seven spacecraft subsequent to launch. Only those requirement changes that resulted from operational (i.e., post-launch) anomalies were of interest here, since the goal was to better understand the relationship between critical anomalies during operations and how safety-critical requirements evolve. The results of the study were surprising in that anomaly-driven requirements changes during operations were rarely due to previous requirements having been incorrect. Instead, changes involved new requirements either (1) for the software to handle rare but high-consequence events or (2) for the software itself to compensate for hardware failures or limitations. The prevalence of new requirements as a result of post-launch anomalies suggests a need for increased requirements-engineering support of maintenance activities in these systems. The results also confirm both the difficulty and the benefits of pursuing requirements completeness, especially in terms of fault tolerance, during development of critical systems.

PLFaultCAT: A Product-Line Software Fault Tree Analysis Tool

Industry currently employs a product line approach to software development and deployment as a means to enhance quality while reducing development cost and time. This effort has created a climate where safety-critical software product lines are being developed without the full range of accompanying safety analysis tools available to software engineers. Software Fault Tree Analysis (SFTA) is a technique that has been used successfully to investigate contributing causes to potential hazards in safety-critical applications. This paper further extends the adaptation of SFTA to product lines of systems by describing a software safety analysis tool called PLFaultCAT. PLFaultCAT is an interactive, partially-automated support tool to aid software engineers in the application of product-line software SFTA. The paper describes the integration of product-line SFTA and PLFaultCAT with the software development life cycle. The description includes the initial construction of the product-line SFTA as well as the automated derivation of software fault trees for product line members. The technique and tool are illustrated with a small case study throughout the paper.