Ganesh J. Pai

Empirical Analysis of Software Fault Content and Fault Proneness Using Bayesian Methods

We present a methodology for Bayesian analysis of software quality. We cast our research in the broader context of constructing a causal framework that can include process, product, and other diverse sources of information regarding fault introduction during the software development process. In this paper, we discuss the aspect of relating internal product metrics to external quality metrics. Specifically, we build a Bayesian network (BN) model to relate object-oriented software metrics to software fault content and fault proneness. Assuming that the relationship can be described as a generalized linear model, we derive parametric functional forms for the target node conditional distributions in the BN. These functional forms are shown to be able to represent linear, Poisson, and binomial logistic regression. The models are empirically evaluated using a public domain data set from a software subsystem. The results show that our approach produces statistically significant estimations and that our overall modeling method performs no worse than existing techniques.

Automatic synthesis of dynamic fault trees from UML system models

The reliability of a computer-based system may be as important as its performance and its correctness of computation. It is worthwhile to estimate system reliability at the conceptual design stage, since reliability can influence the subsequent design decisions and may often be pivotal for making trade-offs or in establishing system cost. In this paper we describe a framework for modeling computer-based systems, based on the Unified Modeling Language (UML), that facilitates automated dependability analysis during design. An algorithm to automatically synthesize dynamic fault trees (DFTs) from the UML system model is developed. We succeed both in embedding information needed for reliability analysis within the system model and in generating the DFT Thereafter, we evaluate our approach using examples of real systems. We analytically compute system unreliability from the algorithmically developed DFT and we compare our results with the analytical solution of manually developed DFTs. Our solutions produce the same results as manually generated DFTs.

AdvoCATE: an assurance case automation toolset

We present AdvoCATE, an Assurance Case Automation ToolsEt, to support the automated construction and assessment of safety cases. In addition to manual creation and editing, it has a growing suite of automated features. In this paper, we highlight its capabilities for (i) inclusion of specific metadata, (ii) translation to and from various formats, including those of other widely used safety case tools, (iii) composition, with auto-generated safety case fragments, and (iv) computation of safety case metrics which, we believe, will provide a transparent, quantitative basis for assessment of the state of a safety case as it evolves. The tool primarily supports the Goal Structuring Notation (GSN), is compliant with the GSN Community Standard Version 1, and the Object Modeling Group Argumentation Metamodel (OMG ARM).

Towards Measurement of Confidence in Safety Cases

Safety cases capture a structured argument linking claims about the safety of a system to the evidence justifying those claims. However, arguments in safety cases tend to be predominantly qualitative. Partly, this is attributed to the lack of sufficient design and operational data necessary to measure the achievement of high-dependability goals, particularly for safety-critical functions implemented in software. The subjective nature of many forms of evidence, such as expert judgment and process maturity, also contributes to the overwhelming dependence on qualitative arguments. However, where data for quantitative measurements can be systematically collected, quantitative arguments provide benefits over qualitative arguments in assessing confidence in the safety case. In this paper, we propose a basis for developing and evaluating the confidence in integrated qualitative and quantitative safety arguments. We specify a safety argument using the Goal Structuring Notation (GSN), identify and quantify uncertainties therein, and use Bayesian Networks (BNs) as a means to reason about confidence in a probabilistic way. We illustrate our approach using a fragment of a safety case for an unmanned aircraft system (UAS).

A Formal Basis for Safety Case Patterns

By capturing common structures of successful arguments, safety case patterns provide an approach for reusing strategies for reasoning about safety. In the current state of the practice, patterns exist as descriptive specifications with informal semantics, which not only offer little opportunity for more sophisticated usage such as automated instantiation, composition and manipulation, but also impede standardization efforts and tool interoperability. To address these concerns, this paper gives (i) a formal definition for safety case patterns, clarifying both restrictions on the usage of multiplicity and well-founded recursion in structural abstraction, (ii) formal semantics to patterns, and (iii) a generic data model and algorithm for pattern instantiation. We illustrate our contributions by application to a new pattern, the requirements breakdown pattern, which builds upon our previous work.

A lightweight methodology for safety case assembly

We describe a lightweight methodology to support the automatic assembly of safety cases from tabular requirements specifications. The resulting safety case fragments provide an alternative, graphical, view of the requirements. The safety cases can be modified and augmented with additional information. In turn, these modifications can be mapped back to extensions of the tabular requirements, with which they are kept consistent, thus avoiding the need for engineers to maintain an additional artifact. We formulate our approach on top of an idealized process, and illustrate the applicability of the methodology on excerpts of requirements specifications for an experimental Unmanned Aircraft System.

Perspectives on software safety case development for unmanned aircraft

We describe our experience with the ongoing development of a safety case for an unmanned aircraft system (UAS), emphasizing autopilot software safety assurance. Our approach combines formal and non-formal reasoning, yielding a semi-automatically assembled safety case, in which part of the argument for autopilot software safety is automatically generated from formal methods. This paper provides a discussion of our experiences pertaining to (a) the methodology for creating and structuring safety arguments containing heterogeneous reasoning and information (b) the comprehensibility of, and the confidence in, the arguments created, and (c) the implications of development and safety assurance processes. The considerations for assuring aviation software safety, when using an approach such as the one in this paper, are also discussed in the context of the relevant standards and existing (process-based) certification guidelines.

Heterogeneous Aviation Safety Cases: Integrating the Formal and the Non-formal

We describe a method for the automatic assembly of aviation safety cases by combining auto-generated argument fragments derived from the application of a formal method to software, with manually created argument fragments derived from system safety analysis. Our approach emphasizes the heterogeneity of safety-relevant information and we show how such diverse content can be integrated into a single safety case. We illustrate our approach by applying it to an experimental Unmanned Aircraft System (UAS).

Dynamic safety cases for through-life safety assurance

We describe dynamic safety cases, a novel operationalization of the concept of through-life safety assurance, whose goal is to enable proactive safety management. Using an example from the aviation systems domain, we motivate our approach, its underlying principles, and a lifecycle. We then identify the key elements required to move towards a formalization of the associated framework.

Automating the Assembly of Aviation Safety Cases

Safety cases are among the state of the art in safety management mechanisms, providing an explicit way to reason about system and software safety. The intent is to provide convincing, valid, comprehensive assurance that a system is acceptably safe for a given application in a defined operating environment, by creating an argument structure that links claims about safety to a body of evidence. However, their construction is a largely manual, and therefore a time consuming, error prone, and expensive process. We present a methodology for automatically assembling safety cases which are auto-generated from the application of a formal method to software, with manually created safety cases derived from system safety analysis. Our approach emphasizes the heterogeneity of safety-relevant information, and we show how diverse content can be integrated into a single argument structure. To illustrate our methodology, we have applied it to the Swift Unmanned Aircraft System (UAS) being developed at the NASA Ames Research Center. We present an end-to-end fragment of the resulting interim safety case comprising an aircraft-level argument manually constructed from the safety analysis of the Swift UAS, which is automatically assembled with an auto-generated lower-level argument produced from a formal proof of correctness of the safety-relevant properties of the software autopilot.