Katerina Goseva-Popstojanova

Architecture-based approach to reliability assessment of software systems

Abstract With the growing emphasis on reuse, software development process moves toward component-based software design. As a result, there is a need for modeling approaches that are capable of considering the architecture of the software and estimating the reliability by taking into account the interactions between the components, the utilization of the components, and the reliabilities of the components and of their interfaces with other components. This paper details the state of the architecture-based approach to reliability assessment of component based software and describes how it can be used to examine software behavior right from the design stage to implementation and final deployment. First, the common requirements of the architecture-based models are identified and the classification is proposed. Then, the key models in each class are described in detail and the relation among them is discussed. A critical analysis of underlying assumptions, limitations and applicability of these models is provided which should be helpful in determining the directions for future research.

Architectural-level risk analysis using UML

Risk assessment is an essential part in managing software development. Performing risk assessment during the early development phases enhances resource allocation decisions. In order to improve the software development process and the quality of software products, we need to be able to build risk analysis models based on data that can be collected early in the development process. These models will help identify the high-risk components and connectors of the product architecture, so that remedial actions may be taken in order to control and optimize the development process and improve the quality of the product. In this paper, we present a risk assessment methodology which can be used in the early phases of the software life cycle. We use the Unified Modeling Language (UML) and commercial modeling environment Rational Rose Real Time (RoseRT) to obtain UML model statistics. First, for each component and connector in software architecture, a dynamic heuristic risk factor is obtained and severity is assessed based on hazard analysis. Then, a Markov model is constructed to obtain scenarios risk factors. The risk factors of use cases and the overall system risk factor are estimated using the scenarios risk factors. Within our methodology, we also identify critical components and connectors that would require careful analysis, design, implementation, and more testing effort. The risk assessment methodology is applied on a pacemaker case study.

Characterizing intrusion tolerant systems using a state transition model

Intrusion detection and response research has so far mostly concentrated on known and well-defined attacks. We believe that this narrow focus of attacks accounts for both the successes and limitation of commercial intrusion detection systems (IDS). Intrusion tolerance, on the other hand, is inherently tied to functions and services that require protection. This paper presents a state transition model to describe the dynamic behavior of intrusion-tolerant systems. This model provides a framework from which we can define the vulnerability and the threat set to be addressed. We also show how this model helps us to describe both known and unknown security exploits by focusing on impacts rather than specific attack procedures. By going through the exercise of mapping known vulnerabilities to this transition model, we identify a reasonably complete fault space that should be considered in a general intrusion-tolerant system.

SITAR: a scalable intrusion-tolerant architecture for distributed services

This paper presents a intrusion tolerant architecture for distributed services, especially COTS servers. An intrusion tolerant system assumes that attacks will happen, and some will be successful. However, a wide range of mission critical applications need to provide continuous service despite active attacks or partial compromise. The proposed architecture emphasizes on continuity of operation. It strives to mitigate the effects of both known and unknown attack. We make use techniques of fault tolerant computing, specifically redundancy, diversity, acceptance test, textitvoting—, as well as adaptive reconfiguration. Our architecture consists of five functional components that work together to extend the fault tolerance capability of COTS servers. In addition, the architecture provides mechanisms to audit the COTS servers and internal components for signs of compromise. The auditing as well as adaptive reconfiguration components evaluate the environment threats, identify potential sources of compromise and adaptively generate new configurations for the system.

Comparison of architecture-based software reliability models

Many architecture-based software reliability models have been proposed in the past without any attempt to establish a relationship among them. The aim of this paper is to fill this gap. First, the unifying structural properties of the models are exhibited and the theoretical relationship is established. Then, the estimates provided by the models are compared using an empirical case study. The program chosen for the case study consists of almost 10,000 lines of C code divided into several components. The faulty version of the program was obtained by reinserting the faults discovered during integration testing and operational usage and the correct version was used as an oracle. A set of test cases was generated randomly accordingly to the known operational profile. The results show that 1) all models give reasonably accurate estimations compared to the actual reliability and 2) faults present in the components influence both components reliabilities and the way components interact.

Statistical non-parametric algorithms to estimate the optimal software rejuvenation schedule

In this paper, we extend the classical result by Huang, Kintala, Kolettis and Fulton (1995), and in addition propose a modified stochastic model to determine the software rejuvenation schedule. More precisely, the software rejuvenation models are formulated via the semi-Markov processes, and the optimal software rejuvenation schedules which maximize the system availabilities are derived analytically for respective cases. Further, we develop nonparametric statistical algorithms to estimate the optimal software rejuvenation schedules, provided that the statistical complete (unsensored) sample data of failure times is given. In numerical examples, we examine asymptotic properties for the statistical estimation algorithms.

Common Trends in Software Fault and Failure Data

The benefits of the analysis of software faults and failures have been widely recognized. However, detailed studies based on empirical data are rare. In this paper, we analyze the fault and failure data from two large, real-world case studies. Specifically, we explore: 1) the localization of faults that lead to individual software failures and 2) the distribution of different types of software faults. Our results show that individual failures are often caused by multiple faults spread throughout the system. This observation is important since it does not support several heuristics and assumptions used in the past. In addition, it clearly indicates that finding and fixing faults that lead to such software failures in large, complex systems are often difficult and challenging tasks despite the advances in software development. Our results also show that requirement faults, coding faults, and data problems are the three most common types of software faults. Furthermore, these results show that contrary to the popular belief, a significant percentage of failures are linked to late life cycle activities. Another important aspect of our work is that we conduct intra- and interproject comparisons, as well as comparisons with the findings from related studies. The consistency of several main trends across software systems in this paper and several related research efforts suggests that these trends are likely to be intrinsic characteristics of software faults and failures rather than project specific.

Estimating Software Rejuvenation Schedules in High-Assurance Systems

Software rejuvenation is a preventive maintenance technique that has been extensively studied in recent literature. In this paper, we extend the classical result by Huang et al. (1995), and in addition propose a modified stochastic model to generate the software rejuvenation schedule. More precisely, the software rejuvenation models are formulated via the semi-Markov reward process, and the optimal software rejuvenation schedules are derived analytically in terms of the reward rate. In particular, we consider the two special cases: steady-state availability and expected cost per unit time in the steady state. Further, we develop non-parametric algorithms to estimate the optimal software rejuvenation schedules, provided that the statistically complete (unsensored) sample data of failure time is given. In numerical examples, we compare two models from the viewpoints of system availability and economic justification, and examine asymptotic properties for the statistical estimation algorithms.

Failure correlation in software reliability models

Perhaps the most stringent restriction in most software reliability models is the assumption of statistical independence among successive software failures. The authors research was motivated by the fact that although there are practical situations in which this assumption could be easily violated, much of the published literature on software reliability modeling does not seriously address this issue. The research work in this paper is devoted to developing the software reliability modeling framework that can consider the phenomena of failure correlation and to study its effects on the software reliability measures. The important property of the developed Markov renewal modeling approach is its flexibility. It allows construction of the software reliability model in both discrete time and continuous time, and (depending on the goals) to base the analysis either on Markov chain theory or on renewal process theory. Thus, their modeling approach is an important step toward more consistent and realistic modeling of software reliability. It can be related to existing software reliability growth models. Many input-domain and time-domain models can be derived as special cases under the assumption of failure s-independence. This paper aims at showing that the classical software reliability theory can be extended to consider a sequence of possibly s-dependent software runs, viz, failure correlation. It does not deal with inference nor with predictions, per se. For the model to be fully specified and applied to estimations and predictions in real software development projects, we need to address many research issues, e.g., the detailed assumptions about the nature of the overall reliability growth, way modeling-parameters change as a result of the fault-removal attempts.

Assessing uncertainty in reliability of component-based software systems

Many architecture-based software reliability models were proposed in the past. Regardless of the accuracy of these models, if a considerable uncertainty exists in the estimates of the operational profile and components reliabilities then a significant uncertainty exists in calculated software reliability. Therefore, the traditional way of estimating software reliability by plugging point estimates of unknown parameters into the model may not be appropriate since it discards any variance due to uncertainty of the parameters. In this paper we propose a methodology for uncertainty analysis of architecture-based software reliability models suitable for large complex component based applications and applicable throughout the software life cycle. First, we describe different approaches to build the architecture based software reliability model and to estimate parameters. Then, we perform uncertainty analysis using the method of moments and Monte Carlo simulation which enable us to study how the uncertainty of parameters propagates in the reliability estimate. Both methods are illustrated on two case studies and compared using several criteria.