Swapna S. Gokhale

Reliability prediction and sensitivity analysis based on software architecture

Prevalent approaches to characterize the behavior of monolithic applications are inappropriate to model modern software systems which are heterogeneous, and are built using a combination of components picked off the shelf, those developed in-house and those developed contractually. Development of techniques to characterize the behavior of such component-based software systems based on their architecture is then absolutely essential. Earlier efforts in the area of architecture-based analysis have focused on the development of composite models which are quite cumbersome due to their inherent largeness and stiffness. In this paper we develop an accurate hierarchical model to predict the performance and reliability of component-based software systems based on their architecture. This model accounts for the variance of the number of visits to each module, and thus provides predictions closer to those provided by a composite model. The approach developed in this paper enables the identification of performance and reliability bottlenecks. We also develop expressions to analyze the sensitivity of the performance and reliability predictions to the changes in the parameters of individual modules. In addition, we demonstrate how the hierarchical model could be used to assess the impact of changes in the workload on the performance and reliability of the application. We illustrate the performance and reliability prediction as well as sensitivity analysis techniques with examples.

A time/structure based software reliability model

The past 20 years have seen the formulation of numerous analytical software reliability models for estimating the reliability growth of a software product. The predictions obtained by applying these models tend to be optimistic due to the inaccuracies in the operational profile, and saturation effect of testing. Incorporating knowledge gained about some structural attribute of the code, such as test coverage, into the time-domain models can help alleviate this optimistic trend. In this paper we present an enhanced non-homogeneous Poisson process (ENHPP) model which incorporates explicitly the time-varying test-coverage function in its analytical formulation, and provides for defective fault detection and test coverage during the testing and operational phases. It also allows for a time varying fault detection rate. The ENHPP model offers a unifying framework for all the previously reported finite failure NHPP models via test coverage. We also propose the log-logistic coverage function which can capture an increasing/decreasing failure detection rate per fault, which cannot be accounted for by the previously reported finite failure NHPP models. We present a methodology based on the ENHPP model for reliability prediction earlier in the testing phase. Expressions for predictions in the operational phase of the software, software availability, and optimal software release times subject to various constraints such as cost, reliability, and availability are developed based on the ENHPP model. We also validate the ENHPP model based on four different coverage functions using five failure data sets.

An analytical approach to architecture-based software reliability prediction

Prevalent approaches to software reliability modeling are black-box based, i.e., the the software system is treated as a monolithic entity and only its interactions with the outside world are modeled. However with the advancement and widespread use of object oriented systems design and web-based development, the use of component-based software development is on the rise. Software systems are being developed in a heterogeneous fashion using components developed in-house, contractually, or picked off-the-shelf and hence it may be inappropriate to model the overall failure process of such systems using the existing software reliability growth models. Predicting the reliability of a heterogeneous software system based on its architecture, and the failure behavior of its components is thus absolutely essential. In this paper we present an analytical approach to architecture-based software reliability prediction. The novelty of this approach lies in the idea of parameterizing the analytic model of the software using measurements obtained from testing. To facilitate this we use a coverage analysis tool called ATAC (Automatic Test Analyzer in C), which is a part of a Software Understanding and Diagnosis System (/spl chi/Suds) developed at Bellcore. We demonstrate the methodology by predicting the reliability of an application called as SHARPE (Symbolic Hierarchical Automated Reliability Predictor), which has been used to solve stochastic models of reliability, performance and performability.

Locating program features using execution slices

An important step towards effective software maintenance is to locate the code relevant to a particular feature. We report a study applying an execution slice-based technique to a reliability and performance evaluator to identify the code which is unique to a feature, or is common to a group of features. Supported by tools called ATAC and /spl chi/Vue, the program features in the source code can be tracked down to files, functions, lines of code, decisions, and then c- or p-uses. Our study suggests that the technique can provide software programmers and maintainers with a good starting point for quick program understanding.

Log-logistic software reliability growth model

The finite-failure non-homogeneous Poisson process (NHPP) models proposed in the literature exhibit either constant, monotonic increasing or monotonic decreasing failure occurrence rates per fault, and are inadequate to describe the failure processes underlying certain failure data sets. In this paper, we propose the log-logistic reliability growth model, which can capture the increasing/decreasing nature of the failure occurrence rate per fault. Equations are developed to estimate the parameters of the existing finite-failure NHPP models, as well as the log-logistic model, based on failure data collected in the form of inter-failure times. We also present an analysis of two data sets, where the underlying failure process could not be adequately described by the existing models, which motivated the development of the log-logistic model.

Reliability simulation of component-based software systems

Prevalent Markovian and semi Markovian methods to predict the reliability and performance of component based heterogeneous systems suffer from several limitations: they are subject to an intractably large state space for more complex scenarios, and they cannot take into account the influence of various parameters such as reliability growth of individual components, dependencies among components, etc., in a single model. Discrete event simulation offers an alternative to analytical models as it can capture a detailed system structure, and can be used to study the influence of different factors separately as well as in a combined fashion on dependability measures. We demonstrate the flexibility offered by discrete event simulation to analyze such complex systems through two case studies, one of a terminating application, and the other of a real time application with feedback control. We simulate the failure behavior of the terminating application with instantaneous as well as explicit repair. We also study the effect of having fault tolerant configurations for some of the components on the failure behavior of the application. In the second case of the real time application, we initially simulate the failure behavior of a single version taking into account its reliability growth. We also study the failure behavior of three fault tolerant systems: DRB, NVP and NSCP which are built from the individual versions of the real time application. Results demonstrate the flexibility offered by simulation to study the influence of various factors on the failure behavior of the applications for single as well as fault tolerant configurations.

Quantifying the closeness between program components and features

Abstract One of the most important steps towards effective software maintenance of a large complicated system is to understand how program features are spread over the entire system and their interactions with the program components. However, we must first be able to represent an abstract feature in terms of some concrete program components. In this paper, we use an execution slice-based technique to identify the basic blocks which are used to implement a program feature. Three metrics are then defined, based on this identification, to determine quantitatively , the disparity between a program component and a feature, the concentration of a feature in a program component, and the dedication of a program component to a feature. The computations of these metrics are automated by incorporating them in a tool ( χ Suds), which makes the use of our metrics immediately applicable in real-life contexts. We demonstrate the effectiveness of our technique by experimenting with a reliability and performance evaluator. Results of our study suggest that these metrics can provide an indication of the closeness between a feature and a program component which is very useful for software programmers and maintainers to better understand the system at hand.

Unification of finite failure non-homogeneous Poisson process models through test coverage

A number of analytical software reliability models have been proposed for estimating the reliability growth of a software product. We present an Enhanced Non-Homogeneous Poisson Process (ENHPP) model and show that previously reported Non-Homogeneous Poisson Process (NHPP) based models, with bounded mean valve functions, are special cases of the ENHPP model. The ENHPP model differs from previous models in that it incorporates explicitly the time varying test coverage function in its analytical formulation, and provides for defective fault detection and test coverage during the testing and operational phases. The ENHPP model is validated using several available failure data sets.

Dependency characterization in path-based approaches to architecture-based software reliability prediction

Prevalent black-box based approaches to software reliability modeling are inappropriate to model the failure behavior of modern, component-based heterogeneous systems. Reliability prediction of applications taking into account their architecture is absolutely essential. The path-based approaches to architecture-based software reliability prediction rely on a fundamental assumption that the successive execution of the components is independent. This leads to very pessimistic estimates of software reliability. In this paper, we describe the dependency characterization which is a major bottleneck in the application of path-based approaches to real-life systems, and propose a way to resolve this issue based on time-dependent failure intensity.

Software reliability analysis incorporating fault detection and debugging activities

The software reliability measurement problem can be approached by obtaining the estimates of the residual number of faults in the software. Traditional black box based approaches to software reliability modeling assume that the debugging process is instantaneous and perfect. The estimates of the remaining number of faults, and hence reliability, are based on these oversimplified assumptions and they tend to be optimistic. We propose a framework relying on rate based simulation technique for incorporating explicit debugging activities along with the possibility of imperfect debugging into the black box software reliability models. We present various debugging policies and analyze the effect of these policies on the residual number of faults in the software. In addition, we propose a methodology to compute the reliability of the software, taking into account explicit debugging activities. An economic cost model to determine the optimal software release criteria in the presence of debugging activities is described. Finally, we present the high level architecture of a tool, called SRSIM, for the purpose of automating the simulation techniques presented.